DEV Community

Discussion on: 60fps Javascript while you stringify, parse, process, compress and filter 100Mbs of data

Collapse
 
gautamkrishnar profile image
Gautam Krishna R • Edited

Off-topic, Another approach to handle resource-intensive calculation would be using worker thread instead of the main thread. developer.mozilla.org/en-US/docs/W.... Cool project btw 😄

Collapse
 
miketalbot profile image
Mike Talbot ⭐ • Edited

Completely agree - I do mention it in the write up above. The issue is often the time taken to serialize and deserialize the workload to the other thread. If you are processing binary then it's totally fine and ideal to do it on a worker thread (because you can use transferrable objects). Structured cloning (to threads) used to be very broken, it's now much better but it's still pretty much only the same speed range as a serialize/deserialize to JSON - which can definitely cause frame loss glitches.

I've been wondering idly about adding threading to js-coroutines, but using it to create the packets so that it doesn't glitch on large payloads - thing is, it's pretty specialist to actually need that other thread, so not sure where the benefit cut off would be...

Collapse
 
caseycole589 profile image
Casey Cole

When your objects are so big you have to lz compress your json before sending it to the client then do segmented uncompresses so you don't crash the browser. Have to do it this way because objects have to be available when user loses connection.

Thread Thread
 
miketalbot profile image
Mike Talbot ⭐

Sounds interesting... Seems like you could do with an uncompress that also yields in that case?

Thread Thread
 
miketalbot profile image
Mike Talbot ⭐ • Edited

@CaseyCole - I've added lz-string compression to the library more details here - not sure it will help in your case, but I'm gonna need it! Thanks for pointing me in this direction...