review priorities and larger goals of code

[njr2128]

Go over open issues and goals in order to best set re-design/re-factoring steps

[njr2128]

areas to consider:

• things this repo does: generates derivatives
  • refactoring has taken place to optimize this process, and we seem to be in a good place in this area
  • one potential change: change the way we file write (to make testing easier)
• other thing it does: make this interactable, so that visualizations can be made easily by others, for examples
  • i.e., analysis, visualization, etc of the text data
  • to go further down this road, we should consult users of this part of it --> sandbox?
  • what are the kinds of analysis and visualizations we want to see and/or enable? (PHS should be part of this)
  • TO DO: look back at user stories, example queries we created in the past (bring Greg up to speed by THC/NJR)
• SHOULD WE DECOUPLE THESE TWO THINGS?

[njr2128]

manuscript-object code should enable ready use/creation of datasets that can then be used for visualization or analysis

[njr2128]

A "package" - Separate place for "i want this data in this form" to use for other things

[gschare]

update.py (the process of generating and writing derivatives) should ideally be a simple, single Python script (with some switches and configuration) that simply requests from the "manuscript object" data structures with various transformations applied, e.g. "an array of all folios rendered in plain text", i.e. allFolios/txt.

Most importantly, the core manuscript object ought not to perform any "actions" (writing files) itself. Rather its entire functionality should be to return Python data structures, which may then be used by some script to actually write that data.
There are design principles guiding why I propose this separation that I won't get into right now. Most salient of these, however, is that while manuscript object itself should be highly flexible, the scripts which use manuscript object must be allowed to ossify to some degree and serve narrow, specific purposes.

[njr2128]

spectrum of visualization purposes/services:

• exploratory
• illustrative
• supportive
• serving as evidence for new inferences or interpretations

[njr2128]

building out jupyter notebooks

• Matthew's notebook
• Terry's old one based on J Reeves
• Greg's version