Skip to content

The Each iterators

Functions derived by Each and its variants work independently on their argument items.

Apart from Case, the map iterators are variations of Each '.

Each

The Each iterator applied to a function f derives a function f' of the same rank as f.

Where f has rank \(N\), you can apply f' with bracket syntax to an argument list of \(N\) conforming lists or dictionaries.

q)/ rank 1
q)til'[2 0 5]
0 1
`long$()
0 1 2 3 4
Where \(N>1\) ,f' iterates f over corresponding items from the list arguments.

q)/ rank 2
q)?'[("abc";"defg";"qrs";"tuvw");"best"]
1 1 2 0
Above, the result is ("abc"?"b";"defg"?"e";"qrs"?"s";"tuvw"?"t"). 
q)/ rank 3
q){x~y rotate asc z}'[("abc";"fed");0 1;("cab";"def")]
10b
Similarly for functions up to rank 8.

Unary functions

Where f is unary, so too is f'. But you can’t apply a unary f' with prefix syntax. Why not? Because…

All derived functions have infix syntax regardless of their rank.

Why is that? Don’t know. Brute fact. Mysteries of the q parser.

So you cannot apply unary f' with prefix syntax (e.g. e.g. count'x – wrong!) because the parser sees count' has infix syntax and looks for a left argument that count' does not and cannot have.

You can however parenthesise the derived function, which captures it as a noun (count'), and you can apply a noun with prefix syntax. So (count')x works.

For the same reason you cannot make count' the left argument of Apply At @ with infix syntax (i.e. count'@x – wrong!), but you can use it as an item of its argument list. 

q)@[count';("quick";"brown";"fox")]
5 5 3
You can also use the each keyword (see below), and that is preferred style.

q)count each ("quick";"brown";"fox")
5 5 3

Summary:

f'[x]      / works: bracket syntax 
f'x        / ERROR: prefix syntax
(f')x      / works: (noun) & prefix syntax
f each x   / preferred: keyword

Binary functions

Where f is binary, good q style applies it with infix syntax as x f'y.

Each-both

Because

  • the each keyword is good style for unary functions
  • most q primitives are unary or binary
  • f' has infix syntax, allowing x f'y

the Each iterator has sometimes been called each-both. This wrongly suggests it is limited to binary functions, which is not true.

Summary:

f'[x;y]   / works: bracket syntax
x f'y     / preferred: infix sytax

Higher-rank functions

Where f has rank 3–8, f' will iterate it through list arguments. Use bracket syntax f'[x;y;z].

q)
q)ssr'[("cow";"sheep";"cat";"dog");("c";"ee";"c";"g");"sobe"]
"sow"
"shop"
"bat"
"doe"

Summary:

f'[x;y;z;..]

Each Left and Each Right

The Each Left \: and Each Right /: iterators are syntactic sugar for the common iterations of binary functions in which either x or y is the same for each iteration.

q)"abc",\:"def"  / Each Left
"adef"
"bdef"
"cdef"

q)"abc",/:"def"  / Each Right
"abcd"
"abce"
"abcf"
The same result can be achieved by projecting the function onto the constant argument and iterating the resulting unary with Each.

p:,["abc"]     / projection
p'["def"]      / bracket syntax
p each "def"   / keyword
(p') "def"     / (noun) prefix

Each Parallel

Where f is a unary (or variadic) function, Each Parallel f': iterates f as a unary through the items of a list or dictionary, dividing the work between secondary tasks.

Where no secondary tasks are configured, Each Parallel is the same as Each.

Multi-threading primitives

The multi-threading primitives feature introduced in kdb+ v4.0 makes Each Parallel redundant for q primitives.

Preferred style is to use the peach keyword.

Summary: 

f':[x]      / works: bracket syntax
f peach x   / preferred: keyword

Each Prior

Where f is a binary function (or rank-2 list or dictionary) 

  f':[y]   <==>   y f' I         , -1_y
x f':y     <==>   y f' enlist[x] , -1_y
Where f is one of these six operators:

f:  +  -  *  %  &    ~
I:  0  0  1  1  0W  (::)

(apart from Match1, right-identity elements for f) otherwise I is first y -1.

Each Prior and vectors

Apart from the six operators, for vector y

f':[y]   <==>   y f'prev y   <==>   (f) prior y

q)mod':[3 4 5]
0N 1 1
q)3 4 5 mod'prev 3 4 5
0N 1 1
q)(mod) prior 3 4 5
0N 1 1


q)prev"abc"
" ab"
q),':["abc"]    / Join Each Prior
"a "
"ba"
"cb"
Applied to a data stucture, Each Prior provides a form of scattered indexing. 
q)show A:4 5#.Q.a
"abcde"
"fghij"
"klmno"
"pqrst"
q)2 1 3 A'prev 2 1 3
" hq"
q)A':[2 1 3]   / A Each Prior
" hq"

Non-vector lists

If the first item of a general list is an atom, the derived function can be applied, provided adjacent items conform.

q)q <' prev q:(3;4;99 100;01b;6;50 51 51;7;8)
0b
0b
00b
11b
00b
000b
111b
0b
q)<':[q] ~ q <' prev q
1b
Where y 0 is not an atom, prev y would return a leading empty list, but for an operator Each Prior substitutes I as above.

q)q < prev q:(99 100;3;4;5)
'length
  [0]  q < prev q:(99 100;3;4;5)
         ^
q)prev q
`long$()
99 100
3
4
q)first q -1
0N
q)<':[q]   / first item is 99 100<0N
00b
11b
0b
0b
This allows functions derived from operators and Each Prior to be applied to matrices. (For a lambda to be applied this way, it must handle an empty right argument.)

First items must have non-zero data types.

If 0=type first y, then I will be () and f':[y] will signal a length error.

Keywords

Four keywords correspond to commonly used functions derived by Each Prior.

deltas   -':
differ  <>': 
prev     -': 
ratios   %':
The keywords are variadic, but use them only as unaries.

These keywords are wrappers for the derived functions and so are also variadic. 

q)a
-2 1 -3 4 -1 2 1 -5 4
q)deltas a                    / unary
-2 3 -4 7 -5 3 -1 -6 9
q)deltas[1000;a]              / binary
-1002 3 -4 7 -5 3 -1 -6 9
Good q style prefers prefix and infix syntax, so use the keywords for unary application; and the derived functions for binary. Write 
q)deltas a                   / unary
-2 3 -4 7 -5 3 -1 -6 9
q)1000-':a                   / binary
-1002 3 -4 7 -5 3 -1 -6 9

Iterator keywords

Keywords each, peach, and prior are not iterators but binary keywords with infix syntax.

Their left arguments are functions.

You can ensure a function is parsed as the argument of a keyword (rather than evaluated) by parenthesising it to make a noun. 

q)M:3 4#til 12
q)(+/) each M
6 22 38
Above, (+/) has noun syntax and the derived function +/ is the left argument of each.

Parentheses are unnecessary where a function’s rank is unambiguous. Above, +/ is variadic; and parentheses stop the parser looking for its arguments. Below, count and sum are unambiguously unary, and need no parentheses.

q)sum each M
6 22 38
q)count each M
4 4 4

The keywords each and peach take unary left arguments; prior a binary.

q)(+)prior til 5
0 1 3 5 7
q){x+y} prior til 5
0N 1 3 5 7

Above, the lambda is unambiguously binary and is parsed as the left argument to prior. But + has a (deprecated) unary form and needs parentheses around it.

Except for lambdas and unary keywords, parenthesise all left arguments of each, peach and prior.

So, count each x but (+) prior x.

Exercises

No peeking.

Attempt each exercise before looking at its answer.

The exercises clarify your thinking. Reading answers does not.

  1. Simplify 3 4+'5 6.

    Answer

    The Add operator has atomic iteration so the Each operator is redundant.

    3 4+5 6

  2. Simplify ("quick";"brown";"fox")?\:"o".

    Answer

    Scalar extension makes the "o" atom equivalent here to "ooo", so Each Left can be replaced with Each. 

    ("quick";"brown";"fox")?'"o"

  3. Explain why count each and count' have different types.

    q)type each(count';count each)
106 104h
    Answer

    Functions derived by Each, such as count' have type 106.

    In contrast, count each projects each onto count and is therefore equivalent to each[count;]. As a projection, it has type 104.

  4. Simplify:

    (,["https://";])each urls
    Answer

    Above, Join , is projected on "https://" and the projection forms the left argument of each. It can be simplified by omitting both the parentheses and the semicolon, but with no increase in clarity. 

    ,["https://"] each urls
    Better: use Each Right. 
    "https://",/:urls

  5. Explain why the expressions below have different results.

    q)(+) prior til 5
0 1 3 5 7
q){x+y} prior til 5
0N 1 3 5 7
    Answer

    The lambda adds 0 (i.e. til[5]@0) to til[5]@-1, which is the null 0N.

    The operator is one of the list of special cases. It adds til[5]@0 to itself.

  6. Below, use KEY to extract secret from cipher.

    q)secret:"mypassword"
q)KEY:neg[1+n]?N:2*n:count secret
q)show cipher:./[N cut(N*N)?.Q.a;1_,':[KEY];:; secret]
"ipstdnynogqxjxwboipo"
"rccwdhdgamjcsweyrudu"
"rzfggdyjamjsomuypcqr"
"awijrkjyzavfgkebpjlg"
"pnrppyqrpveeyzjlexbx"
"ailiexwmmrhlmfuijxzu"
"khaqghfgelmfrilvibfm"
"pntmethjkvfggvbpcyfw"
"iunqbvillxfiqkoehrbw"
"rxwwsesustqqrlznekcs"
"ubfndhkcmhjuxlwnadax"
"hkzhrgxvujbkodvgtlso"
"rwhtrzhvlwfxrfyxfdue"
"tpuhqfyiquhpzgysrlcz"
"nbodbwrtpdnmimuubhdt"
"bofzemgnzqtmkbcptgbs"
"dgmvgkkjzajugcleamao"
"pcwtdgepaoggdusfycuh"
"kmxaeockvwyqtdcphlnm"
"xvwsqmuokcapqhtlpghn"
    Answer

    List cipher is a matrix randomly filled with letters into which secret is inserted at indexes generated from KEY.

    To extract secret from cipher regenerate the indices from KEY with Index Each Right ./:

    q)cipher ./: 1_,':[KEY]
"mypassword"

  7. Lesser (&) is a special case when Each Prior is applied to it. Should Greater (|) also be a special case? Justify your answer.

    Answer

    Because 1b ~ 0|0N the special-case model and the definition x|'prev x of Greater Each Prior both return the same result. So, either

    Yes
    Greater should join Lesser and other comparison operators in the list of special cases.
    No

    Greater Each Prior conforms to the definition pattern and is not a special case.

    |':[x]    <==>    x | prev x

  8. Let M:4 5#20?100. Amend sd:{sum x-y} so that sd prior M returns a result.

    Answer

    sd prior M signals a length error with sd:{sum x-y} because first prev M is`long$().

    sd:{sum x-$[count y;y;0]} allows sd prior M to evaluate.

  1. This special treatment for Match ensures that the first item of the result of differ x is 1b – unless x[0] is the general null! ↩