Ada Lovelace Day 2015

I am very late with Ada Lovelace's Day blogging, again. There are plenty of great mathematically minded women whom we should be celebrating, but since this year I've missed the Infinite Posssibilities Conference, I thought I'd go for a  woman of colour.

I feel very honoured to have been their keynote speaker in 2012 and I reckon that they're absolutely right: "African-American, Hispanic/Latina, and American Indian women have been historically underrepresented in mathematics. In 2002, less than 1% of the doctoral degrees in the mathematical sciences were awarded to American women from underrepresented minority groups." This is very wrong.

So I decided to go for Nyedja Nascimento,  first professor of Agronomy in the state of Paraiba, Brazil.

Nyedja Nascimento  

born in Nov 1925, about to complete 90 years, Nyedja was the first woman to graduate from a college in Paraiba in 1949.  Poor, black and a woman, at a time when only rich men would go to college,  Nyedja had to face all kinds of obstacles. Vivek Nigam sent me this story about her. She also made history being the first female professor of the same institution, the School of Agronomy, which was the first university college of the state, created in 1936. The rest of the Universidade Federal da Paraiba would only start in 1955.

Questions, Problems and Projects: Finding the wood from the trees

I am lucky enough to, sometimes, be contacted by clever young programmers who might want to do something more interesting with their talents  than simply work on the next super-duper-one-of-a-kind-money-making start-up.

OK, it only happened twice, but who knows, maybe it will become a trend, some time?

 I love the idea of thinking about interesting projects for clever people. I mean I normally work from what people are doing already and I try to twist it in the directions of the things I want to do, but  been given the task of thinking of interesting, deep, in principle complicated projects, things that do not have to be ready in one week or two, it's quite wonderful.

When I was a professor in Birmingham, I had a bit more of this good feeling of researching what I wanted, but as a professor one has to worry very much  about making sure that milestones are actually in place, that things do work, at least to a certain extent, as a professor owes it to her students to think up projects that have a high probability of success. Professors also owe this high probability of success to their departments and their funders, so research is much more constrained than one might imagine.

[lovely cartoon of research freedom here, when I find it again]

When someone, who's not a student, who's got a job doing something else, wants to discuss interesting complicated problems or tools (like category theory) just for the fun of doing it, then one has much more freedom to think up projects. So here are some things that I really would like to do, if I was a person of independent means, if I didn't have to earn a living.


1. A more purely logic/categorical project, namely check all possible ways of providing categorical models for Classical Logic and showing their pros and cons.

The best explanation I know of the Curry-Howard isomorphism is the paper by Jean Gallier
http://repository.upenn.edu/cgi/viewcontent.cgi?article=1427&context=cis_reports
(this is constructive logics part 1, part 2 is about Linear Logic is it is also very good,  but not as good)

Jean's is completely about constructive logic, but it explains a little why it doesn't work for classical logic. I don't know anything explicitly written to show it doesn't work for classical logic, but I know a collection of works trying to show that it *partly* works for classical logic: Griffin, Filinski, Parigot, Urban, Herbelin, Selinger,  Wadler, etc.. in http://www.lix.polytechnique.fr/~lengrand/Work/Teaching/MPRI/lecture1.pdf Lengrand gives a short history.

(googling just now I found https://www.cs.tcd.ie/publications/tech-reports/reports.08/TCD-CS-2008-55.pdf, which may or not be good)

Anyways the project here is to decide which pros, cons are more important for ourselves and then develop the favorite approach.

2. A more functional programming project: Why Linear Logic doesn't work as a model of resources for computing in Linear Functional Programming? or rather can we make it work now?

For this I had one specific way of formalizing the use of resources in the project xSLAM, explicit substitutions for a linear abstract  machine

http://www.cs.bham.ac.uk/research/projects/xslam/xslam.html.

the write-up with conclusions of the project is in the summary of achievements, but I do not have crisp version of why it didn't work and I really would like to know that.

Following this path might require learning about Linear Logic, categorical combinators and explicit substitutions, all are great fun.

3. A  language oriented project: A New Unified Lexicon

I worked for almost 9 years at PARC with people like Dick Crouch  and Ron Kaplan (my ex-manager in PARC and the Nuance Lab's founder). There we had a project called Bridge, whose aim was to "translate" English sentences into logic (a variant of first order constructive logic), which I called TIL (for textual inference logic). This used Xerox' proprietary technology, specifically the XLE (Xerox language engine) and the LFG Lexical Functional grammar (Ron's brain child). More recently I wrote and talked about reproducing some of this work, with  different grammars and other components in  Bridges from Language to Logic: Concepts, Contexts and Ontologies.

The project in this case would be to created something like the Unified Lexicon (one of the components of Bridge), which was the theme of this paper
https://docs.google.com/file/d/0ByQvGrHooOoHUElwclFSYS1RNTQ/edit.

The unified lexicon from PARC used WordNet, VerbNet, Cyc and proprietary resources (e.g. the subcategorization lexicon) to build a knowledge base where semantics happens.

My idea would be to use WordNet, Open Multilingual Wordnet (http://compling.hss.ntu.edu.sg/omw/), WordNet's morpho-semantic links (described here https://wordnet.princeton.edu/wordnet/download/standoff/) with SUMO (http://www.adampease.org/OP/), instead of Cyc and maybe Yago or perhaps Wikidata to produce a new, souped-up version of the Unified Lexicon.

Of course this is all real research, so I do not guarantee that  any of this will work as expected, but this is part of the fun, correct?

Now I do have collections of lists of interesting things to do with collaborators already in place. Some of my collaborators don't mind the fact that the lists are always long and keep growing. Some others do mind it and feel that there's too much ADD in my lists. I find that, as I grow old and forgetful, hte lists are very handy.

(The photo, woods  in New Jersey, are artwork from my friend Lalita Jategaonkar Jagadeesan)

Nat@Logic 2015

Invited talk at LSFA2015 given, yay! https://sites.google.com/a/dimap.ufrn…/natalogic-2015/lsfa-x
Slides in slideshare.

Had a great time in Natal with Anna as well. Went to the dunes of Jenipabu, the lagoons and the beaches. Buggy on the dunes, fun stuff, even heard about the cooperative of tourism workers managing the "balsas" (ferries). Mostly the ferries exist for the fun of it. But quite nice that they've organized and shared the work and the money.

Logic and Probabilistic Methods for Dialog

ESSLLI, Barcelona, 10-14th August 2015

Together with Charles Ortiz, I organized a workshop on Logic and Probabilistic Methods for Dialog, at the 2015 European Summer School of Logic, Language and Information, in Barcelona, 3-14th August, 2015. The workshop turned up very well indeed, matching some sort of trend in the the whole summer school towards dialog. Altogether there were four activities focusing on dialogue: an introductory course by Camilo Thorne (Reasoning-based Dialogue Systems), an advanced course by David Traum (Computational Models of Grounding in Dialogue), another advanced course by Nick Asher and Eric McCready (Cooperative and Non-cooperative Discourse in Games ) and our workshop, as well as an evening Invited Lecture by Raquel Fernandez.

Here are the talks in the workshop, together with the slides provided by the lecturers. We now have to decide whether we want to take the trouble of trying to publish any kind of proceedings or not.

 

Where are we now?

I have been working on lexical resources for Portuguese for a while with many friends. Somehow the research strands are looking a bit like the roots of this tree in Parque Laje, in Rio. A preliminary discussion, in the shape of a presentation in here.

Mathematicians in the Interwebs

I should be blogging about useful stuff, like my workshop on Dialog that is coming up soon in Barcelona, as part of ESSLLI 2015. But this week by a coincidence there are two great and very different mathematicians making rounds on the interwebs.

One in the Guardian, the other on the New York Times Magazine. Both profiles are very well-written. The mathematicians also write very well, a big bonus.

The Guardian has John Conway, and Andre Joyal just talked about his category of games in WOLLIC 2015, which I helped to organize. I saw Conway a few times in DPMMS in Cambridge, but never talked to him. He was already a big name and I was a shy young phd student and he soon departed for the US.

But there are several other players that appear in the article that I remember well. Prof Cassell's was the head of department when I arrived. Simon Norton was always in the common room and the backgammon ladder was the center of life, the universe and everything. I kept my distance, of course.

The other mathematician is Terry Tao in the New York Times. I only saw/heard Tao once, when he got a prize from Stanford and was around for several lectures. He was every bit as impressive as I expected.

Two different kinds of mathematicians, different styles indeed.
Love the Conway  humble brag joke, not quite self-deprecating, but almost so  “I do have a big ego! As I often say, modesty is my only vice. If I weren’t so modest, I’d be perfect.”

Love the down-to-earth, we can-do-this style of Tao, "The ancient art of mathematics, Tao has discovered, does not reward speed so much as patience, cunning and, perhaps most surprising of all, the sort of gift for collaboration and improvisation that characterizes the best jazz musicians. Tao now believes that his younger self, the prodigy who wowed the math world, wasn’t truly doing math at all." Tao, I think, then meant to quote Lockheart's Lament, but the quote didn't quite make it, a shame.

But I think I liked best the Charles Fefferman quote on this article "The steady state of mathematical research is to be completely stuck. It is a process that Charles Fefferman of Princeton, himself a onetime math prodigy turned Fields medalist, likens to ‘‘playing chess with the devil.’’ The rules of the devil’s game are special, though: The devil is vastly superior at chess, but, Fefferman explained, you may take back as many moves as you like, and the devil may not. You play a first game, and, of course, ‘‘he crushes you.’’ So you take back moves and try something different, and he crushes you again, ‘‘in much the same way.’’ If you are sufficiently wily, you will eventually discover a move that forces the devil to shift strategy; you still lose, but — aha! — you have your first clue." This just about takes me back to the picture of one of the first posts in this blog.  Indeed,  we fight it!

A very old post from May 2013: Learning to love stats

 Last Friday I went to the Symposium described below at CSLI, Stanford. This was very interesting, but I haven't had a chance to digest the contents, properly. Since there was no webpage and there are no slides/papers from the talks, I'm posting the program here, as an 'aide-memoire'.

The main reason why I'm interested is obvious: probabilistic vector space semantics makes a lot of sense as a substitute for what semanticists call the Prime Semantics of Natural Language (origin of the infamous joke: What's the meaning of life?  life prime) but the probabilistic approach doesn't seem to scale so well as far as logical phenomena is concerned: antonyms seem to appear in similar contexts, not in opposite ones; a tiny small word like "not" seems to have a huge effect in meaning; concepts can crisply imply others, whether or not the probablities are similar, etc... One of the ideas I had during the meeting was that maybe what one needs to do is to use some sort of Glue Semantics logical system, with two-tiers: one where we do composition of meanings using say implicational linear logic and one where we do use the vector spaces for the meanings of the constituents themselves, ie nourn phrases, verb phrases and propositional phrases.

The 2012-2013 Cognition & Language Workshop is pleased to announce a

Symposium on Compositional Vector Space Semantics

featuring

   Stephen Clark, Chung-chieh Shan and Richard Socher

  The emerging field of compositional probabilistic vector space semantics for natural languages and other symbolic systems is being approached from multiple perspectives: language, cognition, and engineering. This symposium aims to promote fruitful discussions of interactions between approaches, with the goal of increasing collaboration and integration.

Schedule of Events:
   9:00 - 9:30      Light breakfast
9:30 - 11:00      Chung-chieh Shan (Indiana)
                        From Language Models to Distributional Semantics
                        Discussant: Noah Goodman

11:15 - 12:45      Richard Socher (Stanford)
                         Recursive Deep Learning for Modeling Semantic Compositionality
                         Discussant: Thomas Icard
  
12:45 - 2:00      Lunch
2:00 - 3:30      Stephen Clark (Cambridge)
                      A Mathematical Framework for a Compositional Distributional Model of Meaning
                      Discussant: Stanley Peters  

3:45 - 5:00      Breakout Groups and Discussion 
5:00 -       Snacks & Beverages

Chung-chieh Shan, University of Indiana

Title: From Language Models to Distributional Semantics

Abstract: Distributional semantics represents what an expression means as a vector that summarizes the contexts where it occurs.  This approach has successfully extracted semantic relations such as similarity and entailment from large corpora.  However, it remains unclear how to take advantage of syntactic structure, pragmatic context, and multiple information sources to overcome data sparsity.  These issues also confront language models used for statistical parsing, machine translation, and text compression. 
Thus, we seek guidance by converting language models into distributional semantics.  We propose to convert any probability distribution over expressions into a denotational semantics in which each phrase denotes a distribution over contexts.  Exploratory data analysis led us to hypothesize that the more accurate the expression distribution is, the more accurate the distributional semantics tends to be.  We tested this hypothesis on two expression distributions that can be estimated using a tiny corpus: a bag-of-words model, and a lexicalized probabilistic context-free grammar a la Collins.

  

Richard Socher, Stanford University

Title: Recursive Deep Learning for Modeling Semantic Compositionality
 

Abstract: Compositional and recursive structure is commonly found in different modalities, including natural language sentences and scene images. I will introduce several recursive deep learning models that, unlike standard deep learning methods can learn compositional meaning vector representations for phrases, sentences and images. These recursive neural network based models obtain state-of-the-art performance on a variety of syntactic and semantic language tasks such as parsing, paraphrase detection, relation classification and sentiment analysis.
   Besides the good performance, the models capture interesting phenomena in language such as compositionality. For instance the models learn different types of high level negation and how it can change the meaning of longer phrases with many positive words. They can learn that the sentiment following a "but" usually dominates that of phrases preceding the "but."Furthermore, unlike many other machine learning approaches that rely on human designed feature sets, features are learned as part of the model.
 

Stephen Clark, University of Cambridge

Title: A Mathematical Framework for a Compositional Distributional Model of Meaning

  

Abstract: In this talk I will describe a mathematical framework for a unification of the distributional theory of meaning in terms of vector space models, and a compositional theory for grammatical types (based on categorial grammar). A key idea is that the meanings of functional words, such as verbs and adjectives, will be represented using tensors of various types. This mathematical framework enables us to compute the distributional meaning of a well-typed sentence from the distributional meanings of its constituents. Importantly, meanings of whole sentences live in a single space, independent of the grammatical structure of the sentence. Hence the inner-product can be used to compare meanings of arbitrary sentences, as it is for comparing the meanings of words in the distributional model.  
There are two key questions that the framework leaves open: 1) what are the basis vectors of the sentence space? and 2) how can the values in the tensors be acquired? I will sketch some of the ideas we have for how to answer these questions.