Skip to content

Stories

jrte edited this page Feb 15, 2023 · 19 revisions

Kick Ass or Chew Dirt?

After 30+ years of coding in more different dialects than I can count I can say, with authority, that all of computing is a boondoggle. And I'm not the first to say so (§14, ©ACM 1977, fair use). With a reliable open source compiler for multidimensional regular expressions there is a way forward that might cut developers a bit of slack and reduce rents in the Cloud. It won't help, but it feels good to get it off my chest. I have a metaphor for my computing career. I see a bunch of guys on their knees and face down in the dirt and I ask "What's up?" and they tell me they're digging a hole. I go away and come back, to tell them all about I've got this thing called a shovel. And they all stand up and start throwing rocks ...

About the Name

I like designing things that are simple and persistent. They're tough and they'll still be here tomorrow, sure as the sun shines. For example, search for ribosome on YouTube and you will find excellent animations of a very small automaton of inexplicable origin at work. It has been replicated persistently without significant (hesitating to say design) changes for a few billion years. It is a perfect metaphor and a fundamental design pattern for pattern-driven automata.

Ribose is the sugar that binds nucleotides in DNA and RNA into sequential blueprints coding for life.

ribose

Hence the name.

A Resonant Quote

I am a naive hobbyist with regard to formal language and automata theory, but I came across a quote recently that resonates strongly with my thoughts on how FSTs fit in with information processing. Pierre Boullier is concerned with natural language processing. In 1998 he prepared a research report outlining his range concatenation framework, which he says

...allows a form of modularity which may lead to the design of libraries of reuseable generic grammatical components [and] can act as a syntactic backbone upon which decorations from other domains (say feature structures) can be grafted.

bicycle-gears

I'm guessing that regular sets would constitute a large part of any such library because like bacteria in the natural ecosystem and finite-state automata in information processing systems they are ubiquitous and do almost all of the work. String them out on another construct like a spine or a stack and they can perform new tricks.

Everything is Hard

A distinguished engineer once told me "Regular expressions are hard." when I tried to turn him on to pattern-oriented programming. I should have sent him here: http://www.gocertify.com/articles/who-invented-the-computer-the-eniac-six for a remedial tutorial on what is hard. It is a quick read but if you're pressed for time just jump down a page or two to the subheading The ENIAC Six and take in the first two (very brief) paragraphs for a quick overview. The accompanying image speaks volumes, but that's just the beginning. ENIAC's successor, EDVAC, was programmed with 'twiddle knobs', an IBM invention. Apparently this eased the task of programming these behemoths, but it probably was still pretty hard.

Everything is hard.

Consider this. When John von Neumann sat down to design EDVAC, the use case looming large in his mind was replacing gymnasiums full of physics PhD candidates and post-docs who were solving differential equations in triplicate in support of some secret military stuff (ssshh, they were building bombs). To cut a long story short he laid down the basic instruction-driven CPU/RAM number-crunching machine model that is still with us, persisting without significant design changes over the past 75 years. We've seen lots of giddy-up, but no essential changes to the basic machine model. We've also seen a long and branching progression of programming languages, compiler technology, and development support tools.

But programming is still hard, of course, because at bottom we are trying to do everything using nothing more than a programmable calculator. Suppose Dr von Neumann had a different use case in mind when he started thinking about computing. Maybe he was thinking about the huge volume of digital media that would have to scanned for personal identifying information after the EU passed their digital privacy laws. In that case he might have put his instruction-driven CPU/RAM thoughts aside and focused on a data-driven model that allowed automata, compiled from syntactic descriptions of specific media containers, to navigate sequential media inputs and identify and act on found artifacts.

Had he continued down this path he might have ended up with something like a hybrid FST/CAM+CPU/RAM model (FiniteStateTransducer/ContentAddressableMemory), which is what ribose is emulating. He would have been delighted to find that almost all of the features required on the CPU/RAM side were trivial, because all of the "logic" required to instruct the CPU was encoded in the inputs. The ENIAC Six would have been delighted to be relieved of twiddle knobs, for the most part, and left to focus their excellent minds on teasing patterns out of collections of information, which their minds have evolved quite naturally to do.

a-b-gears-white

Algorithmic thinking comes naturally too, but in the long view it is a relatively new thing. Some folks think that language evolved to allow us to represent processes to ourselves, and others. How to chip flint to produce a perfect arrow head (it's not easy you know), that sort of thing. The cog in cognition. But I've been coding in algorithmic languages for 30+ years and I'm here to tell you that it is very hard and to ask that you please make it go away.

About this Project

Ribose at current stage is a working proof-of-concept, nothing more. That said, I would like to see it developed further if only to ease the load on all of those hard-working calculators and the developers who have to trick them into working as typewriters (next up? cars!).

There are better ways to do things, although they can be hard to countenance when you have billions of dollars invested in making ever smaller and faster calculators, funding and witnessing the arduous evolution of programming languages and development tools, while reaping generous profits from folks who are willing to pay for this stuff. We are hitting the wall on smaller and faster and running wildly complex layers of hand-crafted software to emulate myriad things that don't relate closely to numeric computation. Perhaps it is not surprising that computing consumes a lot of energy. More energy than all of the UK, they say. Iceland is a popular home to server farms. Microsoft is trying to boil the ocean, running server farms in the dark and formerly cold depths of the North Sea. Someone recently tried to buy Greenland...

There are better ways to do things. Pattern-driven automata might not be your go-to guys for weather prediction or simulating the inner workings of stars but they can accomplish tasks in a wide range of domains way faster, consuming less energy, than traditional computers. Presenting and consuming service APIs, preprocessing for NLP models, text mining, real-time signal processing and process control are prime domains but machine processing in any domain involving sequential data would likely be better served by a pattern/automata orientation. We now have souped-up calculators emulating each other in software so why not emulate a transduction engine and get going with pattern-oriented design and development? It would be a steep slope at the get-go (see Everything is Hard), but only because we haven't invested anything like the time and money spent on compilers and development tools. Ribose is a small first step. It shows, at least, that pattern-driven transduction works and will likely outperform algorithmic processing for a wide range of increasingly common tasks.

Pattern-oriented design and development is more easily staged than traditional software development because it presents a clear and clean separation of concerns between design and implementation. The pattern design stage involves teasing the structures of interest out of the noise and describing them in formal syntactic terms. Once the patterns of interest have been articulated the development stage involves simply mapping features in the pattern to the sequential processing instructions (effectors) required to assimilate the data into the target domain and compiling the mapping into a runtime transducer. Runtime works like this: data tickle transducer tickles effectors tickle target. And you're done. If your neighbor wants to use the pattern to apply the data in a different target domain you can hand it to her(it's open source)and she can get her job done even quicker.

IBM has been going on and on about all of the zigabytes of structured and unstructured data produced in the world every day. IBM, I have two things to say to that: First, unstructured data is not data it is just noise and it has no value. I think you are referring to noisy data sources that embed structured features of interest. Second, algorithmic programming languages force developers to articulate these structures of interest while formulating an algorithm to instruct the CPU thing in the calculator machine. It's a split brain thing. Hurts young developer minds. Pattern identification and articulation can be hard, yes, but much less so than when this necessary activity is confounded with concerns relating to downstream processing.

Computing is a lot of work, for developers and support staff and for numeric calculating devices. Work costs money, consumes a lot of energy and generates a lot of heat. With a little investment IBM could have real (or virtual, it's OK for now) transduction engines up and running in a year, tops. It wouldn't be like OS 360, I promise. You would be first in the world, all over again. A cooler blue, and much greener too (nudge, wink).

Think about it.

A Few Words about Time and Space

This is not about space and time in the cosmologic continuum. Computing costs are measured in discrete spatial (RAM) and temporal (clock cycles, or transitions) dimensions. Also in calories, and rents.

In the ribose test suite there are two transducers:

NilSpeedTest: 5 input equivalence classes, 3 states, 6 transitions
  Tintervals: 30 input equivalence classes, 84 states, 232 transitions

Test runs involve preloading sequential input data into RAM and running each transducer 20 times over the input with output muted. After 2-3 runs the JIT compiler has done its work and the run times stabilize. Taking the last measurement from test series for these two transducers:

NilSpeedTest: 20MB input, 0 bytes output, 63 ms, 317 MB/s
  Tintervals: 1.45MB input, 0 bytes output, 11 ms, 132 MB/s

Q: These are both straight FSTs with almost no engagement of effectors, so what accounts for the almost 3-fold difference in throughput?

A: The transition matrix for NilSpeedTest consumes 60 bytes of RAM spanning at most two 64-byte cache lines and basically lives in the 32kb L1 data cache over the lifetime of the 20 test runs. Not so for Tintervals -- it drives much more traffic over the CPU/RAM bus because its transition function ranges over a much broader swath of RAM (10,080 bytes spanning 158 cache lines).

Which is why CAM would be called for in a hybrid FST/CAM+CPU/RAM machine. Only the transitions would require representation in CAM, whereas transition matrices for non-trivial transductions are typically several 10s or 100s of kilobytes in size and active transitions are often sparsely but widely distributed in RAM. With CAM and a 32-bit key (input,state) mapping to 32-bit encoding of (next-state, effect) the transition function for Tintervals would consume only 232x8 (1856) bytes.

CAM cache efficiency can be maximized for any FST, since fixed enumerations of inputs and states can be selected to ensure that transitions that tend to fire contemporaneously are grouped contiguously in CAM. The information required to determine transition firing patterns may be derived algebraically using techniques developed for pattern analysis or empirically by counting transitions while running a sufficiently robust and representative batch of inputs. Pattern or empirical analysis might reveal cut points where it would be advantageous for a transducer to load a different representation of itself to maintain cache efficiency. This information can then be injected into transduction patterns to direct core effectors of the transduction engine to reorganize transitions in CAM.

Transduction has been a well-known technique for sequential information processing for a lot longer than we have had von Neumann's CPU/RAM beasties. Life itself has been tooling along with it since the get go, literally knitting brilliant zoos and botanical gardens from starshit with every tick of the cosmologic clock. Theorists have beaten it almost to death and it is probably a big yawn for peer reviewers by now. But we have no transduction engines available for the massive volumes of sequential information processing that is going on continuously, tout le monde. We got off on the wrong foot because we had a different sort of use case for building computing machines. Today's use cases are more likely to involve wading through reams of text or other sequential media than solving numeric puzzles.

Q: My CPU monitor shows CPU consumption at 99% from start to finish for both of these transductions, What became of the 60% that didn't contribute to throughput in the slower one?

A: Heat. The CPU clock runs much faster than the clock on the CPU-RAM bus. There is a special instruction called twiddle, alias hurry-up-and-wait, that keeps the CPU warm while it waits for the bus to deliver data from RAM to the L1 data cache.

An FST core in silicon would be several orders of magnitude less complex than the CPU cores driving my 11 year old laptop. CAM in principle is no more technically challenging to produce than RAM. It's the same bits just different wiring, but CAM is not produced in quantity so it seems exotic. CAM is not produced in quantity and we have no commercial FST cores because the Powers That Be insist that all can and must be done with arithmetic and some help moving things about in RAM. They have spent a great big pile of money making CPUs and RAM faster and smaller to maintain the illusion that employing legions of developers to labor with a zoo of florid algorithmic programming languages to instruct von Neumann's programmable calculators is sufficient and the best, only, way to keep up with computing demands.

In collaboration with our sister venture, DeepMind, we have developed solutions that have reduced the amount of energy used to cool our data centres by 30%. Sundar Pichai CEO Alphabet Incorporated

And they will fight tooth and nail to keep things that way. Perhaps Google will find a way to warm our homes with excess heat siphoned from the smart TV, fridge, stove, watches, and sundry other domestic gadgets all around while they record our every word and action preferences (?) and share them with even smarter cloud AIs (!) to help us direct our quotidian busyness. Maximally verbose and complex patterns will be used to encode and decode information for transport between machines to ensure that things stay toasty outside too.

Since 2014, Google’s data centres have been using 50 percent less energy than the industry average through the use of highly efficient evaporative cooling solutions, smart temperature and lighting controls and custom built servers which use as little energy as possible. (ComputerWorld, August 9 2019)

Transduction can be applied to energy as well as to information. For example, highly efficient evaporative cooling solutions are the best way to transduce heat generated by computing machines into the atmosphere.

C'est un transducteur, tout le monde.

sun-gears

Ceci n'est pas un Transducteur

I would like to comment on an emergent design pattern in Clojure and other languages. Not wanting to single out @didier-wenzek in particular but ...

ceci n'est pas un transducteur

This pattern composes a reducing function with a serialization function and is widely applicable, but it is very coarse-grained and I'm calling cultural appropriation on it. To be more useful, the serialization component requires expression of structural properties of input so that input becomes navigable. The reduction component requires a family of reduction functions (effectors) relating to a specific target. The transduction component maps features of interest in the input pattern onto sequences of effectors. The transducer can then navigate instances of source material to direct the application of target effectors to complete the composite reduction function.

A more complete and concise description of the transduction process is obtained when it is expressed as a nested stack of regular patterns mapping raw (eg UTF-8) source input to effectors presented by the transduction target. Pattern-driven transduction will become an effective modality for stream processing when Clojure and other languages admit automata, compiled from open source input patterns extended into specific target domains, as first-order objects. Ginr can serve as an ideal model for compiling these from open source input patterns describing specific media containers. I say 'open source input patterns' because they are each, essentially, an API describing a manifest information source. Pattern in hand, developers can map features of interest into myriad specific target domains to suit their needs without the heavy encumbrance of, for example, the JAX stack for XML.

This brings to mind the problem with widespread use of the transducteur pattern, which is commonly applied to reduce data in object models built from human-readable LR(0) serializations of information designed for machine communication. XML, JSON, that sort of thing. The transducteur pattern cannot be applied in this sort of use case until the document has been parsed and compiled to build the object model (or substantial subset thereof). It may be possible to co-opt the document compiler to serve as the serialization component, so that the transducer runs in lock-step with the compiler, but no matter how you slice it there is a lot of excess work involved. This is cumbersome and a waste of CPU time and considerable energy (sum that). Why not just put a domain-specific transducer on the wire to deserialize and process source information directly, in a single LR(0) pass, and be done?

For example, if the quantities involved in the transducteur example were embedded as attribute values in a particular repeating XML element within a document described by a complex XML schema description (XSD), application of the transducteur pattern would require at least a full parse of the document and navigation of the resulting object model just to obtain the collection of values to be presented to the 'transducteur' instance. A more straightforward approach does not require full a review of the XSD at the design stage, all that is required is to express a regular (or context-free if nested elements must be navigated) pattern describing how to identify and extract the data of interest. This can usually be extended with error correction to realize a transducer, or transduction process involving a composition of cooperating transducers, that can wade through extraneous information in the document to locate and extract information from elements of interest and process this information inline in a single pass over the document text.

This is a Transducer

Here is a simple but demonstrative use case for this sort of thing.

Use case

A developer needs to extract interval times between garbage collection (GC) cycles from GC log files so he can correlate them with contemporaneous metrics from his context-free GC automaton.

Input looks like this:

<?xml version="1.0" ?>

<verbosegc xmlns="http://www.ibm.com/j9/verbosegc" version="214f755db_CMPRSS">

<initialized id="1" timestamp="2021-04-06T01:14:50.767">
  (... lots of stuff)
</initialized>

<exclusive-start id="2" timestamp="2021-04-05T23:12:58.597" intervalms="5243.335">
  <response-info timems="0.105" idlems="0.050" threads="6" lastid="0000000000334700" lastname="BenchmarkThread xml.validation 6" />
</exclusive-start>
<af-start id="3" threadId="0000000000329C80" totalBytesRequested="16" timestamp="2021-04-05T23:12:58.597" intervalms="5243.429" type="nursery" />
<cycle-start id="4" type="scavenge" contextid="0" timestamp="2021-04-05T23:12:58.597" intervalms="5243.468" />
<gc-start id="5" type="scavenge" contextid="4" timestamp="2021-04-05T23:12:58.597">
(... it goes on and on, with a <cycle-start .../> clause for each generational garbage collection cycle)
</verbosegc>

The transducer Tintervals defined below should do the job. Its definition involves hardening the pattern describing the feature of interest by projecting its three tapes (input-tape, effector-tape[parameter-tape]) onto one to flatten it, then replacing each input symbol x with (x | nul), where nul is an out-of-band input signaling no transition defined for current input, and composing the result with an editing transducer that directs the automaton to search for a synchronization point ('<' in this case) whenever it receives a nul signal. The edited pattern is then projected back into the original three tapes.

This is an OK thing to do, as long as the document conforms to its declared schema description (XSD).

interval = (
  '<cycle-start id="' digit+ '" type="scavenge" contextid="' digit+ '" timestamp="' (digit+ ('.:-T':alph))+ digit+ '" '
  ('intervalms="', select[`~interval`] clear)
  (digit, paste)+ ('.', paste) (digit, paste)+
  ('"', out[`~interval` NL])
  space* '/>' (utf8 - '<')*
);

l0 = (interval$0):alph;
l1 = (interval$1):alph;
l2 = (interval$2):alph;

null = (
  (
    ((AnyOrNul* @ interval)$(0 1 2))
  @ ((l0$(0,0))* (l1$(0,,0))* (l2$(0,,,0))*)*
    (nul$(0,0)) (nul* l0* l1* l2*)* 
  )
  ((utf8 - '>')* nul*)* '>' (utf8 - '<')*
);

Tintervals = (
  (null* interval*)* @ (((l2 - NL)$(0,0))* (NL, nl)*)*
):dfamin;

Tintervals$(0,1 2):prsseq `build/patterns/automata/Tintervals.pr`;
Tintervals:save `build/patterns/automata/Tintervals.dfa`;

Ginr compiles Tintervals to a DFA (call it T) with 129 input symbols, 95 states, and 553 transitions. Ribose decomposes this into two compact DFAs F and G so that T = G ○ F*. Then F can be represented as a simple vector mapping Unicode ordinal to equivalence class index, where any two ordinals x,y are equivalent if for all states s, T(x,s) = T(y,s), and G can be represented as a reduced transition matrix mapping (equivalence index, state) -> (state', effector*). This reduces Tintervals to a single-valued transducer with 31 input equivalence classes, 85 states, and 209 transitions.

Let's see if it works...

~/git/jrte$ echo lines words characters = $(wc test-patterns/inputs/verbosegc.vgc)
lines words characters = 23601 103261 1522902 test-patterns/inputs/verbosegc.vgc

~/git/jrte$ java -Djrte.out.enabled=true \
-classpath /home/kb/git/jrte/jars/ribose.0.0.0.jar:/home/kb/git/jrte/jars/ribose.0.0.0-test.jar \
com.characterforming.jrte.test.FileRunner Tintervals patterns/test/inputs/verbosegc.vgc build/Test.model \
>/tmp/j

~/git/jrte$ head -n 4 /tmp/j

Tintervals: 
5243.468
1427.420
2184.707

~/git/jrte$ grep '<cycle-start[^>]*scavenge[^>]*>' test-patterns/inputs/verbosegc.vgc \
| grep -o 'intervalms="[0-9.]*"' | grep -o '"[0-9.]*"' | tr -d '"' \
>/tmp/g

~/git/jrte$ diff /tmp/j /tmp/g
0d1
<Tintervals: 

~/git/jrte$ echo jrte: $((wc -l /tmp/j))
jrte: 524 /tmp/j

~/git/jrte$ echo grep: $((wc -l /tmp/g))
grep: 523 /tmp/g

Yup (the header line in the ribose output accounts for the difference between grep and ribose matches).

Discussion

There are a number of ways to satisfy this simple use case, because the solution involves identifying a simple regular pattern. But most of the available tools (grep, awk, sed, perl, etc) are essentially derivatives of the original work done by Thompson, back in the 70s and they haven't evolved much since then. They present cramped and limited means for expressing patterns and minimal support for interacting with identifiable features, get stuck in tar pits owing to their nondeterministic basis (NFAs express complexity in the temporal domain, wasting time pruning dead-end sideways alternatives), and they cannot deal directly with context-free or context-sensitive inputs.

Ribose can be used in a much broader range of use cases because it admits a transduction stack and target effectors that have access to RAM. Not shown here (another use case, perhaps) but in the ribose runtime transducers can call each other, and themselves, and push out-of-band signals into the input stack (eg inject eol to reduce and replace a complex delimiting sequence) for immediate transduction to guide themselves or to return guidance to the calling transducer. Target can copy prospective snippets out of the input and inject them back into the input stack after looking ahead indefinitely to determine context, allowing oracular resolution of ambiguities. Transducers that have a strictly deterministic basis have linear cost functions unless the target interacts to inject non-linear features (DFAs express complexity in the spatial domain and waste no time triggering effector sequences per input and moving on).

These features extend the range of problems that can be addressed well beyond regex, and solutions are more succinctly expressed as runtime compositions of transducers that cooperate to effect a reduction and assimilation of serial input into a domain-specific target. In this use case, the input was an instance of a specific type of XML document and the target was the transduction itself, sporting its native edit-menu suite of effectors (select, cut, copy, paste, clear), I/O effectors (in, signal, out), control effectors (mark, reset, start, stop) etc, but input need not be textual. Consider series of quantized analog signals, composite multivariate signals, workflow events, any source that generates sequences of discrete things that can be mapped by an equivalence relation R of finite index. The inputs received from the source, if they are of any interest modulo R, will likely have repeating and alternating patterns that can be used to drive transduction processes. The general idea driving ribose development, such as it is, is to make robust transducer technology more widely available. It will save a lot of time and energy in the long run, I think.

Horsefeathers

Any software solution involving significant volumes of text is unnecessarily painful and tedious in C (pick a superscript if you like), Java, Python, you name it, unless it builds on a mile-high stack of libraries and dependencies, some with configuration idiosyncrasies requiring frequent searches for non-existent or outdated guidance, that at bottom look like

   if (c == '\n') {
      ...
   }

all over the place. This burns a lot of oil, is hard to look at, and it is not necessary. Adapting algorithmic languages for process control solutions is another convoluted exercise. They end up looking like

  switch (f(c)) {
  case e0:
    switch (s) {
    case s0:
      // handle event c of class e0=f(c) for state s0 & change state 
      _s0(c);
      s = s1;
      break;
// it goes on and on...

which is a completely opaque description of something that could be expressed much more succinctly as regular patterns in an (event,effect) *-semiring and compiled to a compact transducer expressing the same control logic in terms of event messages and effectors expressed by a process control target. In fact, if you come across source that looks like the above it was probably generated by a regular expression compiler of some sort, proprietary and bound to a limited domain. Any transducer involving more than a handful of states is difficult to directly express and maintain.

New programming languages are popping up all over the place with novel idiomatic adornments and it's getting hard to keep track. It is difficult to watch this progression because most of these horsefeather convolutions are intended to ease the pain of expressing algorithmic solutions for non-numeric use cases like sequential information processing.

Horseshoes

The not-so-hidden agenda behind ribose development is to stand firm on something solid like *-semiring algebra as a primary mode of expression in the design and development of serial data and temporal event processing pipelines. Pattern oriented workflow design involves articulating patterns that syntactically locate and describe artifacts of interest in manifest sequential data sources and mapping them onto semantic effectors that extract and render these artifacts in the consumer domain.

Patterns can be subjected to substantial testing algebraically (eg, conformance to constraints in the effector semiring) and functionally (test pattern recognition in the input semiring using corpora of conformant and nonconformant inputs) in the design stage, before proceeding with development of transduction targets and specialized effectors. Development of target and effectors is tightly focussed in the absence of syntactic concerns. The control structures induced on the target effectors by transduction patterns can be validated, since targets can and should define patterns describing full or partial constraints on firing sequences for their effectors. A file target should allow only E = open (read* write*)* close, for example. Any transduction pattern F targeting a file can then be validated by testing (F$1) ⊆ E. Compilers for classical algorithmic programming languages rigorously validate involved data structures, why not validate control structures as well? Why don't Java interfaces express constraints on sequencing expressed methods? I don't know.

The ribose transduction model presents a clean separation of syntactic (pattern) and semantic (target) concerns. Semiring algebra is stable and, as ginr demonstrates, *-semirings are closed under a rich and robust suite of operators that allow patterns to be articulated, manipulated and tested algebraically to produce rich and robust descriptions of transduction processes. Complex transducers can be expressed directly inline or as collections of nested transducers to be composed on a runtime transduction stack. Target and effector implementation is eased because most of the transduction logic is encoded syntactically in the input. Correctness proofs are much more accessible, and cost functions and other quantitative metrics can be derived directly from generating functions induced by *-semirings patterns.

Last Words?

Ribose is in stasis pending some expression of interest from the community. As it stands it is possible to define and run context-free text transductions against text files, but it really needs some real-world use cases to drive further development. It is also a bit of a red herring, because what I am really driving at is much larger than what I could take on myself.

I believe that transduction technology could be used to

  1. reduce energy consumption for computing by developing hardware support for running FSTs, and
  2. relieve and improve developer minds by teaching a pattern-oriented approach, based on *-semiring algebra, to solving use cases involving sequential information.

Regarding the first point, an FST engine is relatively simple machine since it just conveys (transduces) information onto CPU instructions, no questions asked. Regular and context-sensitive patterns in the input provide the information required to direct the transduction, This obviates the stack of method calls ranging through various layers of hand-crafted software that would push the instruction pointer through a much longer and more complex pattern of instructions to achieve equivalent effect. Sum the energy savings over all the world's data centers and see how that works out.

On the second point, I have three kids and I have successfully steered them away from computing because I think modern programming models ruin people's minds. Computing has a solid basis and *-semiring algebra is a very important, fundamental, component. Kids who cut their teeth on the arithmetic of strings will be much better positioned for solving real-world problems than those chasing the latest fleeting candy offered up in today's programming languages and cumulative software repositories.

Transduction is not that hard to wrap your head around, it just seems foreign because there has not been any real support for *-semirings and transducers in the classical programming domain. Ginr provides necessary support but it is old and a bit cranky. Ribose is a ship in a bottle (not Moby Dick). I will never see *-semiring-v0.8.12. It is just *-semiring algebra and it is as likely to sprout new features as is elementary arithmetic. I like that. Anyone else likes it, consider motherhood for ginr, which appears to be dormant, or ribose (this is a hobby and hail-mary sort of thing for me).

I am retired and, frankly, sick almost to death of coding with traditional software modalities. While I would get behind any effort to bring transduction to the developer masses, I also believe that we would make a significant contribution to reducing world-wide energy consumption if we adopted transduction technology as the primary mode for dealing with use cases involving sequential media. If anyone is interested in moving this forward I would ask them to share their thoughts in the ribose Discussions or contact me directly here: jrte.project@gmail.com.

I'm just gonna say that information is congruent to energy, and let the physicists and philosophers fill in the details. Transduction is a hardcore and cool way to get things done, It just makes sense, if we are going to keep on chasing our informanic tails forever and cool the planet at the same time.