Status of JS version

[RobertSasak]

I am glad to see that project is moving from proof of concept to production. I hope it is OK to ask about the status of Javascript implementation of jsonbinpack that is currently archived at https://github.com/jviotti/jsonbinpack-poc

Is it ok to use?
Will there be a new version?

[jviotti]

Hi @RobertSasak ,

Thanks for reaching out! The version at https://github.com/jviotti/jsonbinpack-poc was the proof of concept submitted with the papers, etc. It mostly works, and feel free to use it.

This repo is meant to hold the proper production ready version of it. However, it is not at feature parity with the JS PoC yet, mainly given other priorities like our schema registry and other similar tooling.

That said, I'd love to hear more about your use case to see if we should be re-prioritising this!

[RobertSasak]

Thank you for repply. I am interested in compressing of large collection of JSON documents. JSON documents have predifined schema and lot of properties are of type ENUM. Simple GZIP compression do not utilize both schema nor enums.

Another option is protobuf which in theory sounds as solution to go. However there are only few semi-experimental tools that can convert JSON schema to probuf schema. Protobuf also has strict naming rules regarding enums and do not allow enums start with number and contain some special characters.

I like your solution as it do not require extra step of creating protobuf schema. Nor it is necessary to handle special cases of enums. Compression rate is comparable but protobuf is slightly better.

ZOD JS is a amazing data validation framwork. It also can produce JSON schema in v4 out of the box. I an imagine JSONBINPACK to work amazing with it.

[jviotti]

@RobertSasak Thanks for the details. Makes sense. What is the schema you are trying to compress with? I don't think we are far from being able to cover a "good enough" subset of JSON Schema, and I could prioritise getting it ready for your use case

[RobertSasak]

It is not publically available schema. However it only consist of nested properties of boolean, number, string, datetime, arrays and enums.

I have JSON validation in ZodJS using z.object, z.string, z.number, z.array and z.enum. I use z.toJSONSchema to generate actual JSON schema. So JSON schema do not contain any of those more advance features.

[jviotti]

Alright, cool! I'll post any updates here. I'll keep the issue open in the mean-time