Day 47: Design: Social Media Feed with Infinite Scroll (Twitter/Instagram)
Design a feed that scales to thousands of items, handles new-content insertion gracefully, and stays smooth via virtualization and pagination.
Study
Concepts
Cursor pagination, not offset pagination
Offset-based pagination (`?page=3&size=20`) breaks under concurrent writes: if 5 new posts are inserted while a user is scrolling, page boundaries shift and items get duplicated or skipped between requests. Cursor-based pagination (`?after=<opaque_cursor>`) anchors each page to a stable position (typically an encoded timestamp+id of the last-seen item) that stays correct even as new content is inserted elsewhere in the feed — this is the standard approach for any high-write-volume, infinitely-scrolling feed.
The feed itself is virtualized (Day 37's techniques, web equivalent: `react-window`/`react-virtual`, or FlashList on mobile) — rendering thousands of real DOM nodes for a long feed is both a memory and a scroll-performance problem, so only visible-plus-buffer items are ever mounted.
New content while the user is scrolled down: don't just insert
Silently prepending new posts to the top of a feed a user has scrolled 40 items into causes a jarring scroll-position jump (everything shifts down). The standard UX pattern is a "N new posts" pill/banner that, when tapped, either scrolls to top or prepends smoothly — new content is fetched and held in a separate buffer, not merged into the rendered list until the user opts in (or the merge is done with a scroll-anchor-preserving technique).
Client-side caching here overlaps directly with Day 28: each feed page is itself cacheable and de-duplicable (the same post might arrive via a "for you" query and a "following" query) — a normalized cache (keyed by post ID, not by which query returned it) prevents inconsistent duplicate copies of the same post from drifting (e.g. one showing stale like-count) when the user has multiple feed views open.
See It
Visualizations
Visualization
Infinite scroll interaction flow
Initial load
Fetch first page via cursor=null
render + virtualize first ~20 items
Scroll near bottom
Prefetch next page using last item's cursor
before the user hits the literal end
New posts arrive server-side
Held in a buffer, NOT auto-inserted
shown as a "12 new posts" pill
User taps the pill
Smoothly merge buffered posts in
preserve or intentionally reset scroll position
Build It
Code Examples
Cursor-based pagination with React Query's infinite query
function useFeed() {
return useInfiniteQuery({
queryKey: ['feed'],
queryFn: ({ pageParam }) =>
fetch(`/api/feed?after=${pageParam ?? ''}`).then((r) => r.json()),
// Server returns { items, nextCursor } — nextCursor stays correct
// even if posts were inserted elsewhere in the feed since last fetch.
getNextPageParam: (lastPage) => lastPage.nextCursor,
initialPageParam: null,
});
}
function Feed() {
const { data, fetchNextPage, hasNextPage } = useFeed();
const posts = data?.pages.flatMap((page) => page.items) ?? [];
return (
<VirtualizedList
data={posts}
onEndReached={() => hasNextPage && fetchNextPage()}
onEndReachedThreshold={0.5} // prefetch before hitting the literal bottom
renderItem={({ item }) => <PostCard post={item} />}
/>
);
}"New posts" buffer instead of a silent, jarring insert
function useFeedWithNewPostsBuffer(latestSeenId) {
const [buffered, setBuffered] = useState([]);
useEffect(() => {
const id = setInterval(async () => {
const res = await fetch(`/api/feed/new?since=${latestSeenId}`);
const newPosts = await res.json();
if (newPosts.length) setBuffered(newPosts); // held, not merged
}, 15_000);
return () => clearInterval(id);
}, [latestSeenId]);
function mergeBufferedPosts(prependFn) {
prependFn(buffered);
setBuffered([]);
}
return { newPostsCount: buffered.length, mergeBufferedPosts };
}Remember
Key Takeaways
- Cursor-based pagination stays correct under concurrent inserts; offset-based pagination silently duplicates/skips items.
- Virtualize the feed — thousands of rendered DOM/native nodes is both a memory and a scroll-jank problem.
- Never silently insert new content above a scrolled-down user — buffer it and offer an explicit "N new posts" affordance.
- A normalized, ID-keyed cache prevents the same post appearing inconsistently across multiple feed queries (e.g. stale like counts).
- Prefetch the next page before the user hits the literal bottom (onEndReachedThreshold) to avoid a visible loading gap.
Do It
Practice
- 1Implement cursor-based pagination against a mock API and prove it stays correct when items are inserted mid-scroll (offset pagination would not).
- 2Build the "N new posts" buffered-banner pattern and wire it to smoothly prepend on tap.
- 3Design (on paper) how you would normalize a feed cache so the same post shown in two different feed queries always reflects the same like count.