• 151

    Projects submitted

  • 109

    Published projects

  • 47

    Comments left

  • 66

    Number of votes

  • 52

    days until voting ends

← Back to list

AI-Ready Open-Source Analytics: SAP S/4HANA on ClickHouse and Superset

  • Project manager on customer side

    Anil Jagetia

    RCM Consumer Products Pvt. Ltd.

    CTO

  • Category

  • Nomination

  • Goals

    Replace fragmented, manual reporting with one governed analytics and AI platform for head office and a network of about 3 million active direct sellers. Before the project, data from SAP S/4HANA and 7 sharded PostgreSQL (Citus) servers sat in silos, reports were slow and built manually, and teams reported different numbers for the same KPI.

    Goals:

    1. consolidate ERP and operational data in one central data lake on self-hosted ClickHouse;

    2. automate ingestion, cleansing, and transformation into curated datasets with one KPI dictionary;

    3. give management self-service dashboards in Apache Superset;

    4. let direct sellers ask questions about their business in natural language and get AI-generated answers, tables and charts;

    5. enable faster, data-driven decisions to optimise inventory and marketing spend;

    6. run the full stack on open-source software, with zero licence fees and full control of the data.

  • Timeline

    november, 2025 — august, 2026
  • Project completion year

    2026

  • Project scale

    2000000 subscribers
  • Results

    The platform is live for head office and the field network, with measurable results:

    • Inventory: down about 17%, from about ₹750 crore to about ₹622 crore, freeing about ₹128 crore of working capital across our 200 warehouses.

    • Marketing: spend cut by about 20%, as campaigns now target the right customers with the right actions.

    • Speed: report and insight delivery about 20x faster; typical queries dropped from about 3 minutes to about 10 seconds on ClickHouse.

    • AI at scale: around 100,000 direct sellers use the Llama-based assistant, with about 500,000 natural-language questions answered, without SQL or IT support.

    • Self-service BI: about 120 Superset dashboards on one governed dataset combining SAP S/4HANA and PostgreSQL data.

    • Cost: zero licence fees; the full stack is open source and self-hosted, keeping all data in-house.

    Next: scaling the AI assistant across our network of about 3 million active direct sellers.

  • Project uniqueness

    As India’s largest direct selling company, with about 3 million active direct sellers a month, we brought data and AI beyond head office to the field: around 100,000 sellers already use the platform, built fully on open source with no licence fees.

    Self-hosted ClickHouse is the central data lake. It consolidates SAP S/4HANA with 7 PostgreSQL servers sharded with Citus, which serve about 10,000 stores and 200,000 home users billing every month, into curated datasets with one KPI dictionary. Queries return in seconds; Apache Superset serves management dashboards.

    On top sits an AI assistant on a self-hosted Llama model. Sellers ask about their business in plain language; the AI generates SQL on ClickHouse and returns answers, tables and charts, with no SQL or IT support. Business data never leaves the company. About 500,000 questions have been answered so far, and the platform is built to scale to the full 3-million network.

  • Software used

    • ClickHouse (self-hosted, open source): central data lake and analytical database

    • SAP S/4HANA: core ERP data source

    • PostgreSQL with Citus extension (open source): 7 sharded servers serving about 10,000 stores and 200,000 home users

    • Apache Superset (open source): BI dashboards and visualisation (about 120 dashboards)

    • Llama (self-hosted, open source): large language model for natural-language questions and SQL generation on ClickHouse

    • Data ingestion / ETL tool,  Airbyte and Kafka data extraction, transformation and loading

    • Server or GPU infrastructure for Llama, self-hosted AI inference

  • Solution from the Global CIO catalog

    The project does not use solutions from the Global CIO catalog

  • Project implementation complexity

    • Heterogeneous sources: SAP S/4HANA’s complex ERP data model had to be combined with 7 PostgreSQL servers sharded with Citus, which handle high-volume transactions from about 10,000 stores and 200,000 home users billing every month.

    • Data consistency: ingestion, cleansing and reconciliation across sources had to produce one trusted KPI dictionary that sales, finance, supply chain and marketing all accept.

    • Performance at scale: data models in ClickHouse were designed so heavy queries return in about 10 seconds instead of 3 minutes.

    • Reliable AI: the Llama model had to turn free-text questions into correct SQL on a large, multi-domain schema and serve 100,000 users with about 500,000 questions.

    • Self-hosting: the whole stack, including the LLM, runs on our own infrastructure, with no vendor support or managed service.

    • Adoption: the solution had to be simple enough for direct sellers in the field with no technical background.

  • Description

    Background As India’s largest direct selling company, we serve about 3 million active direct sellers a month, about 10,000 stores and 200,000 home users who bill every month, supplied from 200 warehouses. Our data was spread across SAP S/4HANA and 7 PostgreSQL servers sharded with Citus. Reports were built manually, heavy queries took about 3 minutes, and teams often reported different numbers for the same KPI. Inventory and marketing decisions relied on slow, partial information.

    Solution We built one governed analytics and AI platform entirely on open-source software, hosted on our own infrastructure.

    1. Data lake: self-hosted ClickHouse is the central analytical store. Data from SAP S/4HANA and the Citus-sharded PostgreSQL cluster is ingested, cleansed and transformed into curated, business-ready datasets with one KPI dictionary, so sales, stock, store and seller data can be analysed together.

    2. Business intelligence: Apache Superset provides about 120 dashboards for management and business teams. ClickHouse’s columnar engine cut typical report queries from about 3 minutes to about 10 seconds, and report and insight delivery became about 20x faster.

    3. Generative AI for the field: an AI assistant based on a self-hosted Llama model lets direct sellers ask questions about their business in plain language. The AI generates SQL on ClickHouse and returns answers, tables and charts, with no SQL skills or IT support needed. Because the model runs on our own servers, business data never leaves the company. Around 100,000 sellers use the assistant, and about 500,000 questions have been answered so far.

    Business impact

    • Inventory reduced by about 17%, from about ₹750 crore to about ₹622 crore, freeing about ₹128 crore of working capital across 200 warehouses.

    • Marketing spend cut by about 20%, as campaigns now target the right customers with the right actions.

    • Faster, data-driven decisions in sales and marketing, based on one trusted source of truth.

    • Zero licence fees: ClickHouse, Superset, PostgreSQL/Citus and Llama are all open source.

    What makes it different Most companies keep BI and AI inside head office and pay for commercial licences per user. We extended analytics and generative AI to a field network of 100,000 sellers at no licence cost, while keeping full control of our data. The same architecture is built to scale across our full network of about 3 million active direct sellers.

    Next steps Scale the AI assistant to more sellers, add new data domains, and introduce forecasting and anomaly detection on the same ClickHouse foundation.

    The forecasting and anomaly detection in “Next steps” are my suggestion, so change them if you have different plans. You have about 400 characters to spare. If you have them, good additions would be the project team size or partner, the implementation timeline, or how you control which data each seller can see.

  • Project geography

    India (nationwide), with head office in Bhilwara, Rajasthan.

    The platform serves operations in every state of India through about 200 warehouses, about 10,000 stores, 200,000 home users who bill every month, and about 3 million active direct sellers a month. Our network reaches deep into rural India, so the solution was designed for field users in small towns and villages, not only in metro cities.

    The full stack (ClickHouse, Superset, PostgreSQL/Citus and the Llama model) is self-hosted in India, so all business data stays under our control and within the country.

  • Additional presentations

Comment

Only authorized users can comment.
We suggest you or register.

  • Customer

    RCM Consumer Products Pvt. Ltd.

We use cookies for analytical purposes and to deliver you the best experience with our website. Continuing to the site, you agree to the Cookie Policy.