Sometimes we need to process data on the front end, perhaps we are using an offline system or accessing local data. When that data gets large it can easily cause the UI to glitch. A few days ago I wrote an article demonstrating how search could be made to run at the same time as UI updates using js-coroutines. I thought I'd dive into a more powerful version in TypeScript that does more than search; it also renders the records as it goes and has a variety of progress indicators. Once done it performs a bunch of tabulations to update some charts.
Notice how you can keep typing and even start browsing the records as the searches continue. This is done using collaborative multitasking on the main thread.
Please note the tooltip supplied by Recharts doesn't work properly when this window is zoomed. See full screen version
This demo uses a new feature of js-coroutines that allows you to define a "singleton" function. Singleton functions automatically cancel the previous run if it is still underway and start again. That's exactly what you need for a search like this.
const process = singleton(function*(resolve: Function, search: string, sortColumn: string) {
let yieldCounter = 0
if (!search.trim() && !sortColumn?.trim()) {
resolve({ data, searching: false })
addCharts(data)
return
}
resolve({ searching: true, data: [] })
let parts = search.toLowerCase().split(" ")
let i = 0
let progress = 0
let output : Data[] = []
for (let record of data) {
if (
parts.every(p =>
record.description
.split(" ")
.some(v => v.toLowerCase().startsWith(p))
)
) {
output.push(record)
if (output.length === 250) {
resolve({data: output})
yield sortAsync(output, (v : Data)=>v[sortColumn])
}
}
let nextProgress = ((i++ / data.length) * 100) | 0
if (nextProgress !== progress) resolve({ progress: nextProgress })
progress = nextProgress
yield* check()
}
resolve({sorting: true})
yield sortAsync(output, (v : Data)=>v[sortColumn])
resolve({sorting: false})
resolve({ searching: false, data: output })
addCharts(output)
function* check(fn?: Function) {
yieldCounter++
if ((yieldCounter & 127) === 0) {
if (fn) fn()
yield
}
}
}, {})
This routine starts off by checking if we are searching for something and takes a quicker path if we aren't.
Presuming it is searching it uses a neat trick of resolving values many times to update the progress. This allows it to reveal results as soon as it has 250 records, update progress every 1% and then switch on and off searching and sorting indicators.
Calling resolve just merges some data into a standard React.useState() which redraws the UI to keep everything smoothly updating while the search progresses.
interface Components {
data?: Array<Data>
searching?: boolean
progress?: number,
sorting?: boolean,
charts?: []
}
function UI(): JSX.Element {
const [search, setSearch] = React.useState("")
const [sortColumn, setSortColumn] = React.useState('')
const [components, setComponents] = React.useState<Components>({})
React.useEffect(() => {
setComponents({ searching: true })
// Call the singleton to process
process(merge, search, sortColumn)
}, [search, sortColumn])
return (
<Grid container spacing={2}>
<Grid item xs={12}>
<TextField
fullWidth
helperText="Search for names, colors, animals or countries. Separate words with spaces."
InputProps={{
endAdornment: components.searching ? (
<CircularProgress color="primary" size={"1em"} />
) : null
}}
variant="outlined"
value={search}
onChange={handleSetSearch}
label="Search"
/>
</Grid>
<Grid item xs={12} style={{visibility: components.searching ? 'visible' : 'hidden'}}>
<LinearProgress
variant={components.sorting ? "indeterminate": "determinate"}
value={components.progress || 0}
color="secondary"
/>
</Grid>
<Grid item xs={12}>
<RecordView sortColumn={sortColumn} onSetSortColumn={setSortColumn} records={components.data} />
</Grid>
{components.charts}
</Grid>
)
function merge(update: Components): void {
setComponents((prev: Components) => ({ ...prev, ...update }))
}
function handleSetSearch(event: React.ChangeEvent<HTMLInputElement>) {
setSearch(event.currentTarget.value)
}
}
The merge function does the work of updating things as the routine progresses, and as we've defined a "singleton" function, it is automatically stopped and restarted whenever the search or sort properties change.
The charts each individually start a calculation, and we "join" their execution to the main process so that restarting the main process will also restart the chart.
function Chart({data, column, children, cols} : {cols?: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12, data: Array<Data>, column: (row: any)=>string, children?: any}) {
const [chartData, setData] = React.useState()
React.useEffect(()=>{
const promise = run(count(data, column))
// Link the lifetime of the count function to the
// main process singleton
process.join(promise).then((result: any)=>setData(result))
}, [data, column])
return <Grid item xs={cols || 6}>
{!chartData ? <CircularProgress/> : <ResponsiveContainer width='100%' height={200}>
<BarChart data={chartData}>
<CartesianGrid strokeDasharray="3 3" />
<XAxis dataKey="name" />
<YAxis />
<Tooltip />
<Bar dataKey="value" fill="#8884d8">
{children ? children(chartData) : null}
</Bar>
</BarChart>
</ResponsiveContainer>}
</Grid>
}
Here we've use a mix of helper Async functions and generators so we have maximum control. Our final remaining generator of interest is the one that calculates the chart results:
function * count(data: Data[], column: (row: Data)=>string, forceLabelSort?: boolean) : Generator<any, Array<ChartData>, any> {
const results = yield reduceAsync(data, (accumulator: any, d: Data)=>{
const value = column(d)
accumulator[value] = (accumulator[value] || 0) + 1
return accumulator
}, {})
let output : Array<ChartData> = []
yield forEachAsync(results, (value: number, key: string)=>{
key && output.push({name: key, value})
})
if(output.length > 20 && !forceLabelSort) {
yield sortAsync(output, (v:ChartData)=>-v.value)
} else {
yield sortAsync(output, (v:ChartData)=>v.name)
}
return output
}
This one simply counts the labels extracted by a function and then sorts the results appropriately.
Top comments (9)
Mike,
Would it be possible to show the charts dynamically updating? After they are done to then show the table?
Excellent work!
BTW I really like the format of your API on the site? What tool was that?
Oh on the API - I used JSDoc and "docdash" for the formatting. Having looked at the code I will post a different version which automatically updates the charts as the search proceeds, it's too different to make the version with this article. It's quite easy, but involves doing the charts "inline"
As it rolled through? Yes that would be possible, you'd just need to be passing the records through to them too I guess.
is this inspired by go courotines?
No, it was kinda inspired by what React are doing with Fiber and the interruptable render - + having used C# coroutines in Unity for a few years.
wow cool i didn't expect you mention c# coroutines here you create game too?
Yes, I was a game programmer for many years (and a very active part of the Unity community between 2011 and 2016).
STOP wasting my computer and phone power for those ridiculous calculations.
Do the job on the server because that's what's its made for !!!
Try doing that offline :) Like I have to. Many applications (like mine) use IndexedDb and work in hostile and offline environments, the user would still like to have a smooth experience and be able to cancel their operations.