1. Data Grid docs
  2. Data

Remote data and infinite scrolling

Connect the grid to your server. Rows load in cached blocks as the user scrolls, the total may be unknown, and search, filters, totals and exports cover the whole result.

When the records live on your server, the grid sends it the whole view (search, filters, sort, groups, columns) and asks for one block of rows at a time. The server answers with the rows of that block, already filtered and sorted. The user scrolls through millions of records while the page holds a few thousand.

Open the server example: 1,000,000 orders, 150 to 400 ms per request

The example's server doesn't report its total until the last block. Add &total=known to the address to see a server that reports it with every block: server example with a known total.

How rows load

  • Blocks of 100 rows at block-aligned offsets (0, 100, 200…). The grid keeps the 24 most recently used blocks and drops the oldest.
  • Prefetch: after a block arrives, the next one is requested too, so steady scrolling rarely waits.
  • Placeholders: rows not yet loaded show as placeholder rows, and the table is marked aria-busy while they load.
  • Settling: a request goes out 90 ms after scrolling stops, so a fast fling past a thousand rows doesn't send a thousand requests.
  • Cancelling: when search, filters, sort, groups or columns change, requests in flight are aborted through their signal and late answers are dropped.
  • Unknown totals: count, matches and sourceCount may be left out. The scroll range then grows a block at a time, the footer says "300+ records", and a block shorter than asked for (or an empty one) marks the end.

Use a REST endpoint

remoteTableSource turns a query URL and optional URLs for export, detail, update and pivots into a data source. Each request is a POST with a JSON body; give headers (an object or an async function) for authentication.

JavaScript

import { mountTable, remoteTableSource } from '@8sheets/data-grid';

const grid = await mountTable(document.querySelector('#orders'), {
  title: 'Orders',
  fields: [
    { name: 'Order', type: 'number', format: '0' },
    { name: 'Customer', type: 'string' },
    { name: 'Region', type: 'string' },
    { name: 'Revenue', type: 'number', format: '#,##0.00' },
  ],
  dataSource: remoteTableSource({
    queryUrl: '/api/orders/query',
    exportUrl: '/api/orders/export',  // enables Export
    detailUrl: '/api/orders/detail',  // the record dialog asks the server
    headers: async () => ({ Authorization: `Bearer ${await getToken()}` }),
    formats: ['xlsx', 'csv'],
  }),
  rowSelection: 'multiple',
});

React

import { useMemo } from 'react';
import { remoteTableSource } from '@8sheets/data-grid';
import { DataGrid } from '@8sheets/data-grid-react';

const fields = [
  { name: 'Order', type: 'number', format: '0' },
  { name: 'Customer', type: 'string' },
  { name: 'Region', type: 'string' },
  { name: 'Revenue', type: 'number', format: '#,##0.00' },
];

export function Orders({ token }) {
  // A new source mounts a new grid, so build it once per token.
  const options = useMemo(() => ({
    title: 'Orders',
    fields,
    dataSource: remoteTableSource({
      queryUrl: '/api/orders/query',
      exportUrl: '/api/orders/export',
      headers: { Authorization: `Bearer ${token}` },
    }),
  }), [token]);
  return <div style={{ height: 600 }}><DataGrid options={options} /></div>;
}

Vue

<script setup>
import { computed } from 'vue';
import { remoteTableSource } from '@8sheets/data-grid';
import { DataGrid } from '@8sheets/data-grid-vue';

const props = defineProps({ token: String });
const fields = [
  { name: 'Order', type: 'number', format: '0' },
  { name: 'Customer', type: 'string' },
  { name: 'Region', type: 'string' },
  { name: 'Revenue', type: 'number', format: '#,##0.00' },
];
const options = computed(() => ({
  title: 'Orders',
  fields,
  dataSource: remoteTableSource({
    queryUrl: '/api/orders/query',
    exportUrl: '/api/orders/export',
    headers: { Authorization: `Bearer ${props.token}` },
  }),
}));
</script>

<template>
  <div style="height: 600px"><DataGrid :options="options" /></div>
</template>

Angular

import { Component } from '@angular/core';
import { DataGridComponent, DataGridOptions } from '@8sheets/data-grid-angular';

// The Angular component loads the grid from your static files, so write the source here: it is the
// same contract remoteTableSource implements.
const post = (url: string, body: unknown, signal?: AbortSignal) =>
  fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(body), signal }).then((r) => {
    if (!r.ok) throw new Error(`Request failed (HTTP ${r.status})`);
    return r;
  });

@Component({
  selector: 'app-orders',
  imports: [DataGridComponent],
  template: `<eight-data-grid [options]="options" style="height: 600px" />`,
})
export class Orders {
  options: DataGridOptions = {
    title: 'Orders',
    fields: [
      { name: 'Order', type: 'number', format: '0' },
      { name: 'Customer', type: 'string' },
      { name: 'Region', type: 'string' },
      { name: 'Revenue', type: 'number', format: '#,##0.00' },
    ],
    dataSource: {
      formats: ['xlsx', 'csv'],
      query: ({ signal, ...request }) => post('/api/orders/query', request, signal).then((r) => r.json()),
      exportResults: ({ signal, ...request }) => post('/api/orders/export', request, signal).then((r) => r.blob()),
    },
  };
}

What the server receives and returns

The query body is the view state plus a window of rows and columns:

{
  "state": {
    "columns": [{ "field": "Order" }, { "field": "Customer" }, { "field": "Region", "pinned": true }, { "field": "Revenue", "total": "sum" }],
    "search": "acme",
    "filters": [{ "field": "Revenue", "op": "gt", "value": 100, "quick": true, "text": ">100" }],
    "sort": [{ "field": "Revenue", "desc": true }],
    "groups": [],
    "collapsed": [],
    "total": null,
    "totalsRow": true
  },
  "offset": 300,
  "limit": 100,
  "columnOffset": 0,
  "columnLimit": 32
}

Apply access control, search, filters, grouping and ordering on the server, then return the slice. The answer is a TablePage:

{
  "offset": 300,
  "rows": [
    { "id": 48213, "values": ["48213", "Acme Ltd", "East", "1,320.00"], "raw": [48213, "Acme Ltd", "East", 1320] }
  ],
  "columns": ["Order", "Customer", "Region", "Revenue"],
  "matches": 12034,
  "count": 12034,
  "sourceCount": 1000000,
  "summary": "",
  "totals": { "Revenue": "4,182,550.00" }
}
  • rows[].id is a stable record ID: selection, editing and the detail panel use it.
  • values are display text, in the order of columns. raw gives the numbers behind them; value rules and the detail panel use it.
  • count counts every row of the result including group rows, matches the matching records, sourceCount the records before filtering. Leave them out while you don't know them.
  • totals holds the display text of each column's totals-row aggregate, over the whole result. stats holds { min, max } for columns with data bars or colour scales.
  • A group row is { group, path, level, collapsed, summary } in place of a record.

The export endpoint gets { state, format } (and a selection when selectionExport is on) and returns the file. The detail endpoint gets { id, state } and returns [{ field, value }] for the record dialog. The update endpoint is described in Editing.

A reference implementation

The source download includes src/table-view/service.js: tableQueryService(table, { editableFields, reuseView }) answers query, update, detail, exportResults and pivot for a table of the 8sheets data engine, including trees, pinned rows and sparkline series. It is what the server example runs, in a Worker, over a million records. Use it as a server in Node for data that fits in memory, or as the reference for the same operations in your database. reuseView: true keeps the last filtered and sorted view between consecutive blocks of one scroll; leave it off when you build a service per request. src/table-view/dev/server.mjs is a complete local HTTP server with queries, detail, export and snapshot links.

Your own source

dataSource can be any object with a query method; update, detail, exportResults and pivot are optional, and pinnedRows: true says its pages answer pinned rows. The query gets { state, offset, limit, columnOffset, columnLimit, signal } and returns a TablePage. Pass signal to fetch, so a cancelled request stops.

const source = {
  async query({ state, offset, limit, signal }) {
    const url = `/api/orders?offset=${offset}&limit=${limit}&q=${encodeURIComponent(state.search)}`;
    const { items, total } = await (await fetch(url, { signal })).json();
    return {
      offset,
      columns: ['Order', 'Customer', 'Revenue'],
      rows: items.map((o) => ({ id: o.id, values: [String(o.id), o.customer, o.revenue.toFixed(2)], raw: [o.id, o.customer, o.revenue] })),
      matches: total, // or leave out while the API doesn't know it
      count: total,
      summary: '',
    };
  },
};

Reload, switch and handle errors

  • refresh() asks the server again for the rows in view, for example after your app changed the data; refresh({ purge: true }) also forgets the known total, as a new query does.
  • setDataSource(source, { fields, title, editable, storageKey }) switches the grid to another source without a new mount.
  • getCounts() gives { matches, sourceCount, count, exact }; matches is null while the server hasn't reported it. getRecords({ offset, limit }) reads matching records through your query, for example to send them elsewhere.
  • A failed block request fires the error event (and the onError option) and shows the message in the grid's status line. The first block of a new source is a load: loadStart and loadEnd fire, and the loading panel shows when it takes longer than 150 ms.

JavaScript

const grid = await mountTable(document.querySelector('#orders'), {
  title: 'Orders',
  fields,
  dataSource: remoteTableSource({ queryUrl: '/api/orders/query' }),
  onError: (err) => toast(`Orders could not be loaded: ${err.message}`),
});

// Another tab changed the data: fetch the visible rows again.
channel.addEventListener('message', () => grid.refresh());

// Archived orders come from another endpoint.
archive.addEventListener('change', (e) =>
  grid.setDataSource(remoteTableSource({ queryUrl: e.target.checked ? '/api/archive/query' : '/api/orders/query' }), { fields }),
);

React

<DataGrid
  ref={ref}
  options={options}
  onError={(err) => toast(`Orders could not be loaded: ${err.message}`)}
/>
// After a save elsewhere in the app:
await ref.current.refresh();

Vue

<DataGrid ref="dataGrid" :options="options" @error="(err) => toast(`Orders could not be loaded: ${err.message}`)" />
<!-- After a save elsewhere in the app: await dataGrid.value.refresh() -->

Angular

// template: <eight-data-grid #grid [options]="options" (gridError)="failed($event)" />
@ViewChild('grid') grid!: DataGridComponent;
failed(err: unknown) {
  this.toast.show(`Orders could not be loaded: ${(err as Error).message}`);
}
async afterSave() {
  await this.grid.refresh();
}

Trees, pinned rows, pivots and live changes

The view state a server gets can hold more than search, filters and sort:

  • tree and treeCollapsed: answer with each record's level, hasChildren and collapsed; see Tree data.
  • pinnedRows, when the source sets pinnedRows: true: answer with pinned: { top, bottom }; see Pinned rows.
  • Columns with a sparkline of other fields: give each record row its series; see Sparklines.

A source with pivot builds pivots on the server (Pivot reports). Moving records only fires rowMove, for your server to reorder them. For data that changes on the server, call refresh() and mark what changed with flashCells() (Live data).

Paged instead of scrolling

scrolling: 'pages' shows 50 records a page with Previous and Next. The footer's Pages and Scroll buttons switch between the two, and so do setScrolling('pages' | 'continuous'). getPage() and setPage(index) read and change the page; count is null while the total is unknown.

Limits

  • The server does the work: search, filters, sorting, grouping and totals are only as complete as its answer. The grid checks the shape of each page and refuses an invalid one.
  • Calculated fields, adding records, the fill handle, Find, transactions and layout exports need local data. Pivots need local data or a source with pivot.
  • getSelectedIds() can't list "every matching record" for a server source; resolve that selection on the server (see Row selection).
  • tableQueryService accepts at most 250 rows or columns per request.