Iterator syntax
DRAFT
Implicit iteration is (almost always) the best iteration in q. For everything else, there are iterators.
Iterators are… amazing. To use them fluently, you need a firm grip on their syntax.
For simple cases, q’s iterators are easier to learn than k’s adverbs. But they introduced some syntactic complexity, which we must now master.
Iterators are unary operators with postfix syntax. That is to say, an iterator has a single argument, which can be written on its left. It returns (or derives) a function, the derived function.
A simple example:
q)("quick";"brown";"fox")?'"ioo"
2 2 1
Above, the derived function Find Each ?' finds
"i"in"quick""o"in"brown""o"in"fox"
The Each iterator ' takes Find ? as its argument, to derive the function Find Each ?'.
This is a thing. It has type 106 and you can Assign it a name.
q)fe: ?' / Find Each
q)type fe
106h
q)fe[("quick";"brown";"fox");"ioo"]
2 2 1
Postfix syntax is conventional, and good q style. However, bracket syntax always works.
q)'[?] [("quick";"brown";"fox");"ioo"]
2 2 1
q)fe ~ ('[?])
1b
To iterate a function that is the result of evaluating an expression, Apply At might be what you need.
q)@[';?] [("quick";"brown";"fox");"ioo"]
2 2 1
q)fe ~ @[';?]
1b
Variadic derived functions
Functions derived from binary functions by the Each Prior, Over and Scan iterators are variadic.
q)+':[1 2 3 4] / Add Each Prior (unary)
1 3 5 7
q)+':[1000;1 2 3 4] / Add Each Prior (binary)
1001 3 5 7
q){x+y}':[1 2 3 4] / {lambda} Each Prior (unary)
0N 3 5 7
q){x+y}':[1000;1 2 3 4] / {lambda} Each Prior (binary)
1001 3 5 7
q)+\[1 2 3 4] / Add Scan (unary)
1 3 6 10
q)+\[1000;1 2 3 4] / Add Scan (binary)
1001 1003 1006 1010
- Each Prior
-
For
r:x f':ythe value ofr[0]isf[y 0;x]. The value ofxis used instead ofy[-1].See Variations on Each for its value when
r:f':[y]. - Accumulators Over and Scan
- For
x f/yandx f\ythe result of the first iteration isf[x;y 0]. - For
f/[y]andf\[y]the result of the first iteration isy[0]. - The value of
xis used as a starting point for the accumulation.
Derived functions are overloaded
As the q Reference tells us, symbols such as ? are overloaded.
The ? symbol denotes several operators, such as Find, Roll, Select and Vector Conditional.
So which of them does ?' denote?
All of them.
The derived function is as variadic as the argument function.
Many binary operators are secretly variadic
In k many binary operators, such as + and &, have unary overloads, which in q are wrapped as keywords. For example, the unary forms of + and # are, respectively, flip and count.
These unary overloads are exposed infrastructure from k, which is undocumented and unsupported. But there all the same.
As a consequence, a derived function such as #', which looks like it should be unambiguously binary, is actually variadic. It is both Take Each and Count Each, and has both binary and unary syntax.
And count', which looks unambiguously unary, has variadic syntax too.
All functions derived postfix have infix syntax
That’s right, all of them.
Regardless of their actual ranks
Some of them have ranks to match.
q)+/[1000;2 3 4] / +/ as binary, bracket syntax
1009
q)1000+/2 3 4 / +/ as binary, infix syntax
1009
Some don’t, e.g. count'.
So how is q to know whether, say, +/ is to be parsed as a binary or a unary?
You tell it.
You have four ways to do that:
-
Bracket syntax, as above, is unambiguous: e.g.
+/[x;y](binary), or+/[x](unary). -
Infix syntax is unambiguous, e.g.
x+/y(binary). -
Name it, and apply it with bracket syntax (unary or binary) or prefix syntax (unary).
-
Parenthesise it, and apply it with prefix syntax (unary).
Project a binary function
To project a binary derived function, include the semicolon. That is, +/[1000;] is a projection, but +/[2 3 4] is not.
Functions derived with bracket syntax do not have infix syntax
For example, +\ is variadic and has infix syntax.
q)100+\1 2 3 / binary, infix
101 103 106
q)+\[100;1 2 3] / binary, bracket
101 103 106
q)+\[1 2 3] / unary, bracket
1 3 6
(\[+]) – also variadic – does not have infix syntax.
q).[~] parse each ("+\\";"(\\[+])")
1b
q)(\[+]) [100;1 2 3] / binary, bracket
101 103 106
q)(\[+]) [1 2 3] / unary, bracket
1 3 6
q)(\[+]) 1 2 3 / unary, prefix
1 3 6
q)100(\[+]) 1 2 3 / binary, infix ERROR
'Cannot write to handle 100. OS reports: Bad file descriptor
[0] 100(\[+]) 1 2 3
^
Naming
Naming a derived function removes its infix syntax.
q)tot: +/
q)tot 2 3 4
9
q)tot[100;2 3 4]
109
You can evaluate 100+/2 3 4 but not 100 tot 2 3 4 because, while tot is variadic, as +/ is, it does not inherit its infix syntax.
But because tot has lost infix syntax, you can apply it as a unary without ambiguity.
That is, you do not have to use parentheses, Apply At, or bracket syntax; you can just write tot 2 3 4.
You can write a lot of q without mastering this, because q provides keywords for the most common derived functions, and also for the iterators themselves.
Parenthesising
Parenthesising a derived function gives it noun syntax.
Now, in the expression it appears in, it’s a thing. It doesn’t have agency. It doesn’t act on anything else. Unless… you apply it.
q)(+/) . (1000;2 3 4) / Apply (binary)
1009
q)(+/) @ 2 3 4 / Apply At (unary)
9
As usual, good q style prefers infix and prefix syntax.
q)1000+/2 3 4
1009
q)(+/) 2 3 4
9
Now you understand why derived functions are parenthesised for unary application.
Keywords for unary application of derived functions
ƒ ƒ/ ƒ\
--------------
+ sum sums
* prd prds
| max maxs
any
& min mins
all
These are not learning aids to be abandoned once you have mastered iterator syntax: good q style prefers the keywords.
Keywords for iterators
Similarly, q has keywords for the iterators themselves.
' each
/ over
\ scan
': prior
These keywords are not themselves iterators. They are binary functions with infix syntax.
q)type(') / iterator
103h
q)type(each) / function
100h
The each keyword takes as its left argument a unary function:
x each y is equivalent to x'[y] or (x')y.
As functions, these keywords do not return derived functions. But you can project them and apply the projection much as you would apply a derived function.
q)ced: count' / count Each (derived function)
q)cep: count each / count Each (projection: each[count;])
q)ced ("quick";"brown")
5 5
q)cep ("quick";"brown")
5 5
Again, good q style prefers count each x to count'[x] or (count')x.
But this has its limits.
q)M:(("quick";"brown");("fox";"jumps"))
q)count''[M]
5 5
3 5
Above, the derived function count' is the argument of the second Each, deriving the function count''.
q)ce0: count'' / count each each
q)ce1: (count')' / postfix syntax: count' is the argument of '
q)ce0 ~ ce1
1b
The two iterators amount to one loop nested inside another.
Using the each keyword would not make this clearer.
q)(count each)each M
5 5
3 5
q)each[count each]M
5 5
3 5
Use keywords for simple expressions; for more complex expressions, master iterator syntax.
Do this for all the iterator keywords: each, over, scan and prior.
Applicable values
The argument of an iterator is an applicable value.
That includes functions, but also lists, dictionaries, and communication handles.
Finite-state machines
You can represent a finite-state machine as a vector in which items are also indices, or a dictionary in which values are keys.1
q)show fsm:-10?10 / finite-state machine
4 3 9 1 5 6 8 0 7 2
q)fsm\[7]
7 0 4 5 6 8
q)show rps:`rock`paper`scissors!`scissors`rock`paper
rock | scissors
paper | rock
scissors| paper
q)1 rps\`rock`paper
rock paper
scissors rock
A data atom cannot be applied or indexed. But an int atom that is also a communication handle can be applied and iterated.
q)(-1')("quick";"brown";"fox");
quick
brown
fox
Two kinds of iterators
Functions derived by
- map iterators
- work independently on the items of their argument/s
- accumulator iterators
- use the result of each iteration as an argument value for the next iteration.
The map iterators are Case and variations of Each '.
The accumulators are forms of Over / and Scan \.
Exercises
FIXME
-
Thanks to Oleg Finkelshteyn for this insight. ↩