Answer by Kamil Maciorowski for How can I use Awk to match multiple lines and then concatenate other fields?

The output of your code clearly shows that the last field of each line is never used. No surprise, as inside 1 Gen 1:1 10 we have 1 as $1, Gen as $2, 1:1 as $3 and 10 as $4; but your code does not use $4 at all.

It seems you want your key to be like $2" "$3, not just $2.

This is an attempt that takes your code and fixes the above problem:

awk '{key = $2" "$3;if (key in vals)    vals[key] = vals[key] "," $4;else    vals[key] = $4;}END {for (k in vals)    print k"\t"vals[k];}' ScriptureTest.txt

If you want to print in the order defined by the first occurrence of each key, then this is one way to do it:

awk '{key = $2" "$3;if (key in vals)    vals[key] = vals[key] "," $4;else    { vals[key] = $4;      sequence[n++] = key; }}END {for (m=0; m<n; m++)    { k = sequence[m];      print k"\t"vals[k]; }}' ScriptureTest.txt

Here is a substantially different approach that aggregates lines until the key changes:

awk 'NR == 1 {key = $2" "$3;oldkey = key;vals = $4;}NR != 1 {key = $2" "$3;if (key == oldkey)    vals = vals","$4;else    { print oldkey"\t"vals;      oldkey = key;      vals = $4; }}END {if (NR>0) print key"\t"vals;}' ScriptureTest.txt

It prints kinda on the fly, so it can be sanely used in a pipe even if its input is never going to end; but it aggregates only neighboring lines with the same key. If your input is in a regular file, then this approach may still be preferred because, in general, it will consume less memory, as it does not need to store data from all lines before printing, it just needs to store data for one key at a time. You will get one line of output for each key, only if the content of the input file is adequately sorted, i.e., if for each key all lines with the key are neighbors. Your example input meets this requirement.