Paginating and Optimising MongoDB Queries

skip() and limit() are the obvious way to paginate, but skip(100000) still walks past 100,000 documents. Range-based pagination keeps every page equally fast.

Range (keyset) pagination

// First page
const page = await posts
  .find({ status: 'published' })
  .sort({ publishedAt: -1, _id: -1 })
  .limit(20)
  .toArray();

// Next page: continue after the last document
const last = page.at(-1);
const next = await posts
  .find({
    status: 'published',
    $or: [
      { publishedAt: { $lt: last.publishedAt } },
      { publishedAt: last.publishedAt, _id: { $lt: last._id } },
    ],
  })
  .sort({ publishedAt: -1, _id: -1 })
  .limit(20)
  .toArray();

await posts.createIndex({ status: 1, publishedAt: -1, _id: -1 });

_id breaks ties so documents published at the same time aren't skipped or repeated. Encode publishedAt and _id into an opaque cursor for your API.

Return only what you need

posts.find(filter, { projection: { title: 1, slug: 1, publishedAt: 1 } });

Projections reduce network transfer and memory, and can make queries covered by an index.

Find slow queries

// Log operations slower than 100 ms on this database
await db.command({ profile: 1, slowms: 100 });
const slow = await db.collection('system.profile').find().sort({ ts: -1 }).limit(10).toArray();

Atlas users get the same information in the Query Profiler and Performance Advisor, which also suggests indexes.

Other quick wins

• Avoid unanchored regex searches (/term/) on large collections; use a text or Atlas Search index.
• Avoid $where and JavaScript expressions in queries.
• Keep documents from growing without bound (move ever-growing arrays into their own collection).
• Use countDocuments with a filter sparingly on huge collections; estimatedDocumentCount is instant for totals.