Maybe it’s time we invent JPUs (json processing units) to equalize the playing field.
The best I can do is an ML model running on an NPU that parses JSON in subtly wrong and impossible to debug ways
You need to make sure to remove excess whitespace from the JSON to speed up parsing. Have an AI read the JSON as plaintext and convert it to a handwriting-style image, then another one to use OCR to convert it back to text. Trailing whitespace will be removed.
Did you know? By indiscriminately removing every 3rd letter, you can ethically decrease input size by up to 33%!
Just make it a LJM (Large JSON Model) capable of predicting the next JSON token from the previous JSON tokens and you would have massive savings in file storagre and network traffic from not having to store and transmit full JSON documents all in exchange for an “acceptable” error rate.
Hardware accelerated JSON Markov chain operations when?
So you’re saying it’s already feature complete with most json libraries out there?
Latest Nvidia co-processor can perform 60 million curly brace instructions a second.
Finally, something to process “databases” that ditched excel for json!
60 million CLOPS? No way!
JSON and the Argonaut RISC processors
Until then, we have simdjson https://github.com/simdjson/simdjson
Well, do you have dedicated JSON hardware?
Please no, don’t subsidize anything Java-Script. It will only make it less efficient.
And thus JsPU was born from Lemmy
NOOOOOOOO
Modern sites: This page requires a JsPU to run which is not present on this system. The website will run in reduced feature mode.
4 Js-core @ 2.00 GHz
Slap a liquid cooler and you’re cooking at a high speed 2.08 GHz
My thoughts on software in general over the past 20 years. So many programs inefficiently written and in 4th level languages just eats up any CPU/memory gain. (Less soap box and more of a curious what if to how fast things would be if we still wrote highly optimized programs)
Answer: there’d be far less software in the world, it would all be more archaic and less useful, and our phones and laptops would just sit at 2% utilization most of the time.
There’s an opportunity cost to everything, including fussing over whether that value can be stored as an int instead of a double to save 8 bits of space. High level languages let developers express their feature and business logic faster, with fewer bugs, and much lower ongoing maintenance costs.
I fully concur. There’s tons of really inefficient software out there that wastes resources just because for a long time, available resources grew fast enough to just keep using more of them without the net speed of an application slowing down. If we didn’t have so many lazy SW devs, I suspect the reduction in needed CPU cycles would have a measurable positive effect on climate change.
Even if our apps used resources better the adware will just use the free space.
@Randelung @seaQueue well, i have dedicated JavaScript hardware (https://developer.arm.com/documentation/dui0801/h/A64-Floating-point-Instructions/FJCVTZS)
The R in ARM and RISC is a lie.
The website title says “Arm Developer”, not “ARM Developer”, in a clearly non-acronym way so it’s a guide for making prosthetic hardware. Of course you want a cyborg arm to parse JS natively, why else even get one?
Nope it’s still a register-register op, that’s very much load-store architecture.
It’s reduced, not minimalist, otherwise every RISC CPU out there would only have one instruction like decrement and branch if nonzero. RISC-V would not have an extension mechanism. The instruction exists because it makes things faster because you don’t have to do manual bit-fiddling over 10 instructions to achieve a thing already-existing ALU logic can do in a single cycle. A thing that isn’t even javascript-specific (or terribly relevant to json), it’s a specific float to int cast with specific rounding and overflow mode. Would it more palatable to your tastes if the CPU were to do macro-op fusion on 10(!) instructions to get the same result?
Lie starts with L, dummy
Everybody gangsta still we invent hardware accelerated JSON parsing
https://ieeexplore.ieee.org/document/9912040 “Hardware Accelerator for JSON Parsing, Querying and Schema Validation” “we can parse and query JSON data at 106 Gbps”
106 Gbps
They get to this result on 0.6 MB of data (paper, page 5)
They even say:
Moreover, there is no need to evaluate our design with datasets larger than the ones we have used; we achieve steady state performance with our datasets
This requires an explanation. I do see the need - if you promise 100Gbps you need to process at least a few Tbs.
Imagine you have a car powered by a nuclear reactor with enough fuel to last 100 years and a stable output of energy. Then you put it on a 5 mile road that is comprised of the same 250 small segments in various configurations, but you know for a fact that starts and ends at the same elevation. You also know that this car gains exactly as much performance going downhill as it loses going uphill.
You set the car driving and determine that, it takes 15 minutes to travel 5 miles. You reconfigure the road, same rules, and do it again. Same result, 15 minutes. You do this again and again and again and always get 15 minutes.
Do you need to test the car on a 20 mile road of the same configuration to know that it goes 20mph?
JSON is a text-based, uncompressed format. It has very strict rules and a limited number of data types and structures. Further, it cannot contain computational logic on it’s own. The contents can interpreted after being read to extract logic, but the JSON itself cannot change it’s own computational complexity. As such, it’s simple to express every possible form and complexity a JSON object can take within just 0.6 MB of data. And once they know they can process that file in however-the-fuck-many microseconds, they can extrapolate to Gbps from there
Render the json as polygons?
It’s time someone wrote a JSON shader.
Ray TraSON
Rayson
Someone just needs to make a GPU-accelerated JSON decoder
That is sometime the issue when your code editor is a disguised web browser 😅
CPU vs GPU tasks I suppose.
GPU, render my 4.2 MB json file!
Rockstar making GTA online be like: “Computer, here is a 512mb json file please download it from the server and then do nothing with it”
Reject MB, embrace MiB.
Reject MiB, call it “MB” like it originally was.
C++ vs JavaScript
it’s more like gpu vs CPU
Works fine in
vim
Except if it’s a single line file, only god can help you then. (Or running
prettier -w
on it before opening it or whatever.)cat file.json | jq
also worksRender Media works the best
rm file.json
4.2 megs on one line? Vim probably can handle it fine, although syntax won’t be highlighted past a certain point
Technically every JSON file is a single line, with line break characters here and there
I have the same problem with XML too. Notepad++ has a plugin that can format a 50MB XML file in a few seconds. But my current client won’t allow plugins installed. So I have to use VS Code, which chokes on anything bigger than what I could do myself manually if I was determined.
Time to train an LLM to format XML and hope for the best
Do we need a “don’t parse xml with LLM” copypasta?
L arge L regex M odel
there are simd accelerated json decoders
every day we stray further from god
Don’t worry, they still make extensive use of regexes.
deleted by creator
yea we need multithreaded json parsers
CUDA accelerated JSON parser is sorely needed