Optimizing Mandi Price Latency: Edge Caching Strategies for Rural India
If you are building for the Indian farmer, your biggest enemy isn't the competition; it's the 1500ms round-trip time on a patchy 4G connection in a rural mandi.
At KhetiSetu, we’ve spent a significant amount of time optimizing how commodity prices (Mandi Bhav) reach the end-user. For a farmer standing in a crowded market in Unnao or Mandsaur, trying to negotiate with a trader (Arhatiya), a loading spinner is a failure. They need to know the price of mustard or wheat now, not after a ten-second delay while our server in Mumbai queries a database.
The challenge with mandi data is that it is both highly volatile and geographically specific. Prices fluctuate throughout the morning as arrivals peak, and then stabilize in the afternoon. Fetching this data directly from government APIs or our own scrapers on every request is an architectural bottleneck.
The Architecture of a Low-Latency Price Feed
In a standard web setup, a request travels from the user's phone to the origin server. In rural India, where "last-mile" connectivity is often a misnomer, we have to move the data as close to the user as possible. This means leveraging the Edge.
We use a layered caching strategy that combines Incremental Static Regeneration (ISR) and Edge Middleware. This ensures that while the data is fresh enough to be accurate, it is served from a CDN node in a city near the farmer, rather than a central data center.
Data Flow Overview
graph LR
A[Farmer App] --> B[Edge CDN Node]
B -- Cache Hit --> A
B -- Cache Miss/Stale --> C[Next.js ISR Worker]
C --> D[Redis Cache]
D --> E[Mandi Data Source / API]
E --> D
D --> C
C --> B
By placing the logic at the edge, we reduce the Time to First Byte (TTFB) significantly. A farmer in Punjab hitting a POP (Point of Presence) in Delhi will always have a better experience than hitting a server in a different region.
Implementing ISR for Commodity Prices
Next.js provides a robust way to handle this via ISR. Since mandi prices don't change every second—usually updates happen every 30 minutes to an hour—we don't need real-time WebSockets. We need "fresh enough" data.
Here is a simplified version of how we handle the price fetching logic for specific mandis.
// pages/mandi/[state]/[district]/[commodity].tsx
import { GetStaticProps, GetStaticPaths } from 'next';
export const getStaticProps: GetStaticProps = async (context) => {
const { state, district, commodity } = context.params!;
try {
// Fetching from our internal optimized microservice
const res = await fetch(`https://api.khetisetu.com/v1/prices/${state}/${district}/${commodity}`, {
next: { revalidate: 1800 } // Revalidate every 30 minutes
});
const data = await res.json();
if (!data) {
return { notFound: true };
}
return {
props: {
priceData: data,
lastUpdated: new Date().toISOString(),
},
// Ensure the page is regenerated at most every 30 minutes
revalidate: 1800,
};
} catch (error) {
console.error("Mandi Data Fetch Error:", error);
return { props: { error: true }, revalidate: 60 };
}
};
Why 1800 seconds?
Setting the revalidate timer is a balancing act. If we set it too low, we hammer our upstream sources (like Agmarknet or private scrapers) and increase our compute costs. If we set it too high, the farmer sees "stale" prices that might have been superseded by a mid-day market correction. 30 minutes is the "sweet spot" we discovered through field testing.
Handling the Vernacular and Geographic Long-Tail
India has over 7,000 regulated mandis. When you multiply that by hundreds of commodities and varieties (e.g., Desi vs. Hybrid Maize), you get a massive matrix of potential URLs.
Pre-rendering every single combination is impossible—it would blow up our build times. Instead, we use a "blocking" fallback strategy. When a farmer looks for a niche mandi in a remote part of Odisha that hasn't been cached yet, the first request will be slow as the server generates the page, but every subsequent user in that region will get the cached version instantly.
| Strategy | Latency (2G/3G) | Server Load | Data Freshness |
|---|---|---|---|
| Client-side Fetch | 3000ms+ | High | Real-time |
| Server-side Rendering | 1500ms - 2000ms | High | Real-time |
| Edge Caching (ISR) | 200ms - 500ms | Low | ~30 mins old |
The "Stale-While-Revalidate" UX
One of the nuances of building for rural users is managing their perception of speed. If a page takes 2 seconds to load, the user might think the app is broken and close it.
By using the stale-while-revalidate pattern, we serve the old price data immediately while the background process fetches the new data. However, we must explicitly show the "Last Updated" timestamp.
In the UI, we implement a simple indicator:
- Green: Updated less than 1 hour ago.
- Yellow: Updated 1-4 hours ago.
- Red/Grey: Data is from yesterday (common for mandis that haven't reported yet).
This transparency builds trust. At KhetiSetu, we’ve found that farmers value the "Last Updated" timestamp almost as much as the price itself. It tells them whether they are looking at the Bhav from this morning's auction or yesterday's closing.
Infrastructure Lessons from the Field
When we first deployed, we relied on a single global cache. We quickly realized that the "Average Latency" metric is a lie. Your average might look great because of users in Bangalore on fiber, but your P99 for a farmer in rural Bihar is what actually matters for retention.
We shifted to a regionalized edge strategy. We also implemented a "Cache Warming" script for the top 500 most active mandis. Every morning at 10:00 AM (when the first major price bulletins start dropping), a worker-bot hits the most popular mandi pages. This ensures that when the farmers check their phones during their tea break, the cache is already "warm" and the response is near-instant.
Conclusion
Caching isn't just about saving on AWS bills; it’s a critical component of accessibility. In the Indian agri-tech space, performance is a feature that directly impacts a farmer's livelihood. By moving mandi price feeds to the edge and using intelligent revalidation patterns, we can provide a seamless experience that works even when the network bars are low.
Building for the next billion users requires us to stop thinking about "modern web standards" in a vacuum and start thinking about the physical reality of the user. For KhetiSetu, that reality is a dusty market, a cheap smartphone, and the need for data that arrives before the trader walks away.