Transduction->Induction->Understanding

There are differing opinions on where AI/ML solutions are and where humans are as far as being able to accomplish a set of tasks. It is true that there are tasks that humans currently do better than AI/ML solutions. What I take issue with are articles that forecast the abilities of AI/ML solutions many years away and readers of these articles take some sort of comfort that their livelihood will be protected for X number of years.

One way to assess and compare AI/ML solutions and human’s abilities to solve tasks (complex tasks), is to apply benchmarks. The image, shown above, illustrates applying metrics, to compare AI/ML solutions and human’s ability to solve language understanding. A complete list of benchmark results can be found here. What is interesting is that human baselines were listed as #14 in 2020 and in 2023, human baselines are rated #23. One can argue that human’s rank dropped because of the increasing successes of Transformer networks. Or one can argue the human’s rank dropping is a reflection of the ever-decreasing test scores measuring human understanding of language (there is more to this than just the pandemic). Or possibly the human drop is the transition to a TikTok world. In any case, AI/ML solutions continue to push ahead of average human understanding of language.

Metrics used in determining rank are accuracy score, F1-score, and Matthews Correlation Coefficient.

Accuracy score is about measuring a true or positive result.

F1-scores apply weighted averages for precision and recall.

Matthews Correlation Coefficients (MCC) computes true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN).

There are differences between how AI/ML (Transformer) solutions and humans learn how to understand language.

From Wikipedia:

“In logic, statistical inference, and supervised learning, transduction or transductive inference is reasoning from observed, specific (training) cases to specific (test) cases. In contrast, induction is reasoning from observed training cases to general rules, which are then applied to the test cases. The distinction is most interesting in cases where the predictions of the transductive model are not achievable by any inductive model. Note that this is caused by transductive inference on different test sets producing mutually inconsistent predictions.”

Humans learn to understand language via transduction and inference. Transduction is about learning from patterns or events. If you wake up in the morning and have breakfast, the first event is waking up in the morning. The second event is that you have breakfast. When a child experiences hundreds of events of waking up and having breakfast, the child makes an inference that when it wakes up, it will have breakfast – an induction. The events that the child experiences are collected from vision, smell, and hearing. From these two events, the child learns the structure of language by speaking at the breakfast table. It is a trial-and-error learning experience. First gurgling, Dada, Mama, and finally, more orange juice please.

Machines also apply transduction and induction to understand language. Just as a child learns by seeing the same events, machines can learn by analyzing streams of images (Convolutional Networks)- each image being an event.

Machines leverage Transformer Networks and apply transduction and induction to understand language. Some Transformer networks just have an encoder, and some Transformer Networks have both an encoder and a decoder. Very similar to a child experiencing events of waking up and having breakfast, encoders focus attention on words – events. Surrounding words and position of words have an impact on the inference drawn of what the word means. And much like the human brain, Transformer Networks can focus on multiple words in parallel.

In conclusion, education does not stop in grade, middle, high school or college. Benchmarks clearly show the increasing functionality of AI/ML solutions over shorter and shorter time spans. Ignore the warm and fuzzy posts. If you want to remain relevant, education is key.