bdec/README at master · TauPan/bdec · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158

----
bdec
----

'bdec', a set of tools for decoding binary files.

.. contents::


Overview
========

Writing decoders for binary formats is typically tedious and error prone.
Binary formats are usually specified in text documents that developers have
to read if they are to create decoders, a time consuming, frustrating, and
costly process.

While there are high level markup languages such as ASN.1 for specifying
formats, few specifications make use of these languages, and such markup
languages cannot be retro-fitted to existing binary formats. 'bdec' is an
attempt to specify arbitrary binary formats in a markup language, and create
decoders automatically for that binary format given the high level
specification.

Bdec can;

 * Allow specifications to be easily written, updated, and maintained.
 * Decode binary files directly from the specification.
 * Generate portable, readable, and efficient C decoders.
 * Run under Windows & Unix operating systems

The bdec xml specification uses constructs based loosely on those found in
ASN.1.


Getting started
===============

1. Download_ the latest version.
2. Go through the tutorial_.
3. Try writing a specification for your own file formats (referring to the
   documentation_ where necessary).
4. Save lots of time and pain by *not* writing and maintaining hand written
   buggy decoders.


Documentation
=============

If you're just getting started, the tutorial_ takes you step by step through
the different concepts used in bdec. Once you've got started, refer to the
rest of the documentation_ as required.

For convenience, `downloadable pdf documentation`_ is also available.

.. _documentation: docs/
.. _tutorial: docs/tutorial.html
.. _downloadable pdf documentation: files/bdec-VERSION.pdf


Support
=======

Any comments or questions can go to the group mailing list at
`bdec-project@yahoogroups.com`_. This is a moderated list, so please be
patient if your question doesn't show up immediately.

.. _bdec-project@yahoogroups.com: mailto:bdec-project@yahoogroups.com


Features
========

* It is easy to quickly write (and read) a specification
* No need to write a custom decoder (all error conditions are handled
  internally; decoding either passes or fails).
* The command line application allows visual inspection of problematic files.
* Unicode aware, little-endian / big-endian aware
* Decode to xml for rapid development (eg: `xml representation of a fat
  bootsector`_)
* Specify fields down to the bit (eg: single bit flags)
* Generated decoders are human readable, easy to use, and compile without
  warnings.

.. _xml representation of a FAT bootsector: docs/files/fatbootsector.xml


License
=======

The bdec library and all decoders generated by the specification compiler
are available under the `GNU Lesser General Public License (LGPL)`_. In
summary, this means;

 * The decoders can be built and used in any application (including commercial
   closed source applications).
 * If used in a closed source application, the decoder must be dynamically
   linked (eg: .dll in Windows, .so in Linux).
 * The source code (and any modifications) to the decoder must be made
   available to anyone you distribute the decoder to.

.. _GNU Lesser General Public License (LGPL): http://www.gnu.org/licenses/lgpl.txt


Download
========

The easiest way to install bdec is to use python's easy_install_ application.
Run::

  easy_install bdec

to install the latest version of the bdec decoder and compiler. The source
code links below can be also be used if you install the dependencies manually
(currently pyparsing_ and mako_).

See the CHANGELOG_ for past releases.

.. _CHANGELOG: changelog.html
.. _easy_install: http://pypi.python.org/pypi/setuptools#installation-instructions
.. _pyparsing: http://pyparsing.wikispaces.com/
.. _mako: http://www.makotemplates.org/


Source
======

The source control tree is available on github at http://github.com/asdf1011/bdec.

  git clone http://github.com/asdf1011/bdec.git


Similar projects
================

Some projects that perform 'generic' binary decoding include;

* TSN.1_ - Commercial product with very similar decoding functionality to bdec.
  Can create C, C++, and Java decoders and encoders.
* Encodix_ - Commercial product focussed towards decoders for the
  telecommunications industry.
* Hachoir_ - Very powerful decoding library, where binary formats are specified
  in python code (using generators). Quite high level decoding, with complex
  binary files being decoded. Lots of file formats are specified (at least
  partially).

Projects that generate decoders from high level specifications;

* Asn1c_ Asn.1 compiler; generates C language encoders and decoders.
* `Google protocol buffers`_ - Define file formats in an easy to read text
  specification, and compile them to Java, C++, or Python decoders and encoders.

.. _TSN.1: http://www.protomatics.com/tsn1.html
.. _Hachoir: http://hachoir.org/
.. _Encodix: http://www.dafocus.com/
.. _Asn1c: http://lionet.info/asn1c/
.. _Google protocol buffers: http://code.google.com/apis/protocolbuffers/