User:JamesCrook/Interactive Diagrams

The Idea

The idea is for a block based language for creating interactive diagrams. This will be easier to use than raw JavaScript in the same way that languages like Scratch, Blockly and Snap make programming easier for beginners. I expect this block-based language to work, because interactive diagrams really are programs underneath. Blocks provide a way to repackage JavaScript, with idioms designed for diagram production.

The block editor would run in the browser and produce JavaScript code. Editors would not be able to insert raw JavaScript.

One difference between this planned block-based language and existing block-based languages is that the blocks will also be used for assembling images in a click together way. My hope is that there will rarely be a need to edit a text definition of an image or to edit visually on the final resulting image, because most of the time the block based representation will be adequate for doing either.

The Development Plan

This is an ambitious project. It will take a long time to write and it will be a long time before Wikipedia is willing to accept such an extension. In order to be compelling there will need to be a demo site, possibly using an existing Wikipedia affiliated site, possibly something new. Early stages of development will not be impressive. My plan is to get the early stages into actual use via the open source project I work on, Audacity. This will be relevant particularly for graphs, e.g. for explaining concepts such as Fourier transforms. It is imperative that there is early stage actual use, in order to refine the concepts and the interface and motivate development steps.

Audacity is already experimenting with interactive diagrams. We have a souped-up image map for the program user interface at wit.audacityteam.org. Currently its main use is in producing many static image maps for our online manual - see for example Toolbars Overview. There is already an outline plan within the Audacity team to use blocks inside the Audacity user interface to arrange and organise things, and also for explaining code, with the aim of making it easier for new developers to join in.

IDE

I am writing a block based editor from which you can 'run' the diagram, the same approach that Scratch offers. Whilst not a true IDE, I've seen children in CoderDojo work with and debug reasonably large programs and animations in Scratch without a proper 'debugger language' - so I am fairly confident that this 'editor rather than IDE' approach will work.

From a pedagogical point of view, NOT having a 'debugger language' that is distinct from the diagram language is an advantage. There is less that has to be learned to use the tools.

Texty Operations

If you 'fade out' the backgrounds of the blocks you are left with a text based language. I plan to provide that 'fade to text' option. Blocks are just an accessible presentation. The programs will in fact be stored as text. Diff on the textual presentation will be available.

The textual version of the program will probably be shorter than the equivalent SVG or SMIL, making the diffs less cluttered than SVG or SMIL diffs would be.

Wikimedia Integration

Wikipedia currently supports gifs, SVG and SMIL whereas the block based editor will be using canvas. In its early incarnations the block based editor will be a wikimedia plug in used at a different site. It will function as a souped-up online image editor, separate from wikipedia itself, for producing images to upload to Wikipedia. There will be wrapper templates that can generate to formats that Wikipedia does support.

In particular a large slippy-map style chart of biochemical pathways can be used to generate to interactive 'snapshots'. These will have a restricted chosen set of capabilities, for example supporting functionality to light up particular pathways, but not supporting the zooming for more detail of the full canvas version.

Case Studies

These ideas can get very abstract indeed. In order to make a plan that will work, we need concrete examples of diagrams that benefit from such an approach. It is important to choose families of diagrams, because that is where we get the most bang for our buck. A single program or annotation method or augmentation of a diagram can get used in multiple contexts and so the work done on creating the code is amortised over more diagrams.

Biochemical Pathways

There is already an excellent site, Alex Pico's wikipathways, for wiki-based editing of biochemical pathways. This is designed for researchers. Wikipedia could benefit from something like it that also integrate images into the pathways. Those images should be interactive and take you to a more detailed article. In this case study I would like to look at how to redesign the presentation of biochemical pathways to make them more appealing to earlier learners.

I think it is useful to use experience from the production of underground metro maps. In these we want to be able to show the entire network and also to show a particular pathway. In the London underground, the carriages have a linear map of just the current line, showing the order of stops and the cross connections. The image below is the current non-interactive image used in Wikipedia. I think it's rather busy, and it would benefit from a changed annotation mechanism.

One way to make this image less busy would be to start out with a single pathway in colour and the other pathways faded out to gray. Annotations could depend on the zoom level. For example, if zoomed out maybe only the labels for the Citric Acid Cycle and Urea Cycle would be shown. As we zoom in, labels could become hover labels or there could be labels round the edges that link in to the actual points of interest. Then as we zoom in further individual metabolites could be labelled, and zooming further still we could see the actual molecules. Clicking on the molecules would take us to a page about that molecule

The diagram needs a way to filter so that we highlight a particular pathway whilst still showing it in context. We also need to consider how such a diagram can be collaboratively edited. We need some algorithm that will allow the general shape of the pathways to be designed whilst the fine detail can automatically lay itself out and change e.g. as more compounds are added into the diagram.

Cladograms

Wikipedia already has a template for cladograms. There is also a wonderful ‘Tree of Life’ at the One Zoom site. I don't think the One Zoom presentation is a perfect fit for Wikipedia. Whilst it has some nice interactivity and is very clever in how it is infinitely extensible, I think a not so good aspect is that it does not show how heavily populated sub-branches are until you actually zoom into them. This cladogram is from the Vertebrates page. With interaction we ought to be able to switch to an annotated spindle diagram.

Vertebrata/

Agnatha/ Hyperoartia (lampreys) Myxini Cyclostomes ?†Euconodonta unnamed †Pteraspidomorphi ?†Thelodonti unnamed ?†Anaspida unnamed †Galeaspida unnamed ?†Pituriaspida †Osteostraci Gnathostomata †Placodermi (armoured fishes) unnamed Acanthodians, incl. Chondrichthyes (cartilaginous fishes) Euteleostomi Actinopterygii (ray-finned fishes) Sarcopterygii (lobe-finned fish) ?†Onychodontiformes Actinistia (coelacanths) unnamed †Porolepiformes Dipnoi (lungfishes) unnamed †Rhizodontimorpha †Tristichopteridae Tetrapoda

Craniata

An interactive Wikipedia cladogram could have nodes that you open and close. Again, like biochemical pathways, it could have methods to filter so that you can show up a particular region of the cladogram that is of interest to you or, for example, highlight endangered species

Graphs

Gnuplot already provides a way to create a large family of graphs and maintain the design in a textual definition. d3.js does something similar and provides interactivity

The audacity context drives a particular focus on presentation of time series, where it is important to be able to zoom in to different time scales. Zoomed out, one is essentially in a view that summarises. For explaining audio algorithms, including explaining Fourier transforms, animated gifs such as this one work well:

It would nice be nice to make such a diagram more interactive. A suitably designed block-based language should make producing such a plot much easier than it is currently. Related animations that help explain Fourier transforms can be found at learning websites such as ‘Better Explained’.

Something much more flexible than Gnuplot and easier to use than d3 could be created. I think the annotation methods that could be designed are interesting. It should be easy to use click-together blocks to add a title or to add annotations to an axis or a key or to annotate individual points on a graph or indeed a region. One can learn something of how images can be assembled from components by looking at heraldry. The language describing a shield allows its reconstruction from the components. So too with diagrams that have multiple component plots.