Pyparsing Infixnotation Into A Parse Tree: Only One Operator Ending Up In Tree?

My eventual goal is to apply sql-where-clause-style queries to filter pandas dataframes. Some searching led me to pyparsing's infixNotation method. I found an example of infix not

Solution 1:

Start by decorating your parse action with pyparsing's traceParseAction diagnostic deccorator.

@traceParseActiondefaddAnd():
    exprStack.append("and")

traceParseAction will show the matched source line, the starting location of the matched tokens, and tokens passed to the parse action, and the value that is returned by the parse action:

>>entering addAnd(line: 'cheese and crackers and wine', 0, 
                 ([(['cheese', 'and', 'crackers', 'and', 'wine'], {})], {}))
<<leaving addAnd(ret: None)

The tokens are a little confusing-looking, since what you are getting is a pyparsing ParseResults object, which has both list and dict semantics, so the Python repr of the object first shows its list contents, and then its named contents. What looks like a tuple with a list and a dict is really the ParseResults, and in this case, it is a ParseResults whose first element is another ParseResults, and this nested object is the one containing your list of matched tokens.

This is a little easier to see if you add a tokens argument to your parse action, and then print out tokens.dump():

def addAnd(tokens):
    print(tokens.dump())
    exprStack.append("and")

And you'll get the more readable:

[['cheese', 'and', 'crackers', 'and', 'wine']]
[0]:
  ['cheese', 'and', 'crackers', 'and', 'wine']

You can see that the matched tokens contains not just 'and', but all of the and-ed terms together, so you will need to push as many 'and's onto your exprStack as there are in the matched tokens.

defaddAnd(tokens):
    exprStack.extend(tokens[0][1::2])

With this change, you should now see this as your returned stack:

cheese
crackers
wine
and
and