Skip to content

Support thousands separator in fmtnum #1499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Feb 18, 2024
Merged

Support thousands separator in fmtnum #1499

merged 2 commits into from
Feb 18, 2024

Conversation

@johnkerl
Copy link
Owner

@johnkerl johnkerl commented Feb 18, 2024 •
edited
Loading

For #1488 and #1490

  • Needs documentation updates
  • Needs unit-test cases

@johnkerl johnkerl marked this pull request as draft February 18, 2024 18:18
@johnkerl
Copy link
Owner Author

johnkerl commented Feb 18, 2024 •
edited
Loading

@lulunac27a @aborruso here are some (draft) examples:

$ export LANG=en
$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*1.e9), "%18_d"); $sy = fmtnum($x * 1e9, "%7.4_f")'
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| a   | b   | i  | x                   | y                   | si                 | sy               |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  |      1,000,000,000 | 346,790,144.3381 |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  |      1,414,213,562 | 758,679,964.7900 |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 |      1,732,050,807 | 204,603,305.7663 |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 |      2,000,000,000 | 381,399,393.8711 |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  |      2,236,067,977 | 573,288,919.8020 |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 |      2,449,489,742 | 527,126,160.0919 |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  |      2,645,751,311 | 611,784,060.5678 |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   |      2,828,427,124 | 598,554,009.1064 |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  |      3,000,000,000 | 31,441,876.4609  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  |      3,162,277,660 | 502,626,005.5412 |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+

$ export LANG=fr
$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*1.e9), "%18_d"); $sy = fmtnum($x * 1e9, "%7.4_f")'
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| a   | b   | i  | x                   | y                   | si                 | sy               |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  |      1 000 000 000 | 346 790 144,3381 |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  |      1 414 213 562 | 758 679 964,7900 |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 |      1 732 050 807 | 204 603 305,7663 |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 |      2 000 000 000 | 381 399 393,8711 |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  |      2 236 067 977 | 573 288 919,8020 |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 |      2 449 489 742 | 527 126 160,0919 |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  |      2 645 751 311 | 611 784 060,5678 |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   |      2 828 427 124 | 598 554 009,1064 |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  |      3 000 000 000 | 31 441 876,4609  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  |      3 162 277 660 | 502 626 005,5412 |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+

$ export LANG=it
$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*1.e9), "%18_d"); $sy = fmtnum($x * 1e9, "%7.4_f")'
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| a   | b   | i  | x                   | y                   | si                 | sy               |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  |      1.000.000.000 | 346.790.144,3381 |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  |      1.414.213.562 | 758.679.964,7900 |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 |      1.732.050.807 | 204.603.305,7663 |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 |      2.000.000.000 | 381.399.393,8711 |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  |      2.236.067.977 | 573.288.919,8020 |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 |      2.449.489.742 | 527.126.160,0919 |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  |      2.645.751.311 | 611.784.060,5678 |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   |      2.828.427.124 | 598.554.009,1064 |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  |      3.000.000.000 | 31.441.876,4609  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  |      3.162.277.660 | 502.626.005,5412 |
+-----+-----+----+---------------------+---------------------+--------------------+------------------+

Idea:

  • The %'d is too hard to escape with shell quotes -- use %_d for Miller instead
  • Likewise %_f
  • Language is "en" by default; consults the (very common) LANG environment variable if available
  • I haven't written on-line doc materials yet

What do you think?

@johnkerl johnkerl force-pushed the kerl/fmtnum-commas branch 2 times, most recently from 5831afa to a60348c Compare February 18, 2024 18:51
@aborruso
Copy link
Contributor

aborruso commented Feb 18, 2024

This is really a great new feature. I have no notes to add, only thank you very much!!!

@johnkerl johnkerl marked this pull request as ready for review February 18, 2024 18:56
@johnkerl johnkerl changed the title Support thousands separator in fmtnum [WIP] Support thousands separator in fmtnum Feb 18, 2024
@johnkerl johnkerl merged commit 7bd460a into main Feb 18, 2024
@johnkerl johnkerl deleted the kerl/fmtnum-commas branch February 18, 2024 19:01
@johnkerl johnkerl mentioned this pull request Feb 18, 2024
Closed
@lulunac27a
Copy link

lulunac27a commented Feb 18, 2024 •
edited
Loading

Numeric format with fixed number of decimal digits should be right aligned, not left aligned by default (referenced from this code above.

@johnkerl
Copy link
Owner Author

johnkerl commented Feb 19, 2024

@lulunac27a I'm confused. The examples above do exhibit right-alignment.

@johnkerl
Copy link
Owner Author

johnkerl commented Feb 19, 2024

@lulunac27a I think I see -- ?

$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%18_d")'
+-----+-----+----+---------------------+---------------------+--------------------+
| a   | b   | i  | x                   | y                   | si                 |
+-----+-----+----+---------------------+---------------------+--------------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  |                 10 |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  |                141 |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 |              1,732 |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 |             20,000 |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  |            223,606 |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 |          2,449,489 |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  |         26,457,513 |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   |        282,842,712 |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  |      3,000,000,000 |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  |     31,622,776,601 |
+-----+-----+----+---------------------+---------------------+--------------------+

$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")'
+-----+-----+----+---------------------+---------------------+-------------+
| a   | b   | i  | x                   | y                   | si          |
+-----+-----+----+---------------------+---------------------+-------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  | 10          |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  | 141         |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 | 1732        |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 | 20000       |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  | 223606      |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 | 2449489     |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  | 26457513    |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   | 282842712   |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  | 3000000000  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  | 31622776601 |
+-----+-----+----+---------------------+---------------------+-------------+

$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%_d")'
+-----+-----+----+---------------------+---------------------+----------------+
| a   | b   | i  | x                   | y                   | si             |
+-----+-----+----+---------------------+---------------------+----------------+
| pan | pan | 1  | 0.3467901443380824  | 0.7268028627434533  | 10             |
| eks | pan | 2  | 0.7586799647899636  | 0.5221511083334797  | 141            |
| wye | wye | 3  | 0.20460330576630303 | 0.33831852551664776 | 1,732          |
| eks | wye | 4  | 0.38139939387114097 | 0.13418874328430463 | 20,000         |
| wye | pan | 5  | 0.5732889198020006  | 0.8636244699032729  | 223,606        |
| zee | pan | 6  | 0.5271261600918548  | 0.49322128674835697 | 2,449,489      |
| eks | zee | 7  | 0.6117840605678454  | 0.1878849191181694  | 26,457,513     |
| zee | wye | 8  | 0.5985540091064224  | 0.976181385699006   | 282,842,712    |
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059  | 3,000,000,000  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864  | 31,622,776,601 |
+-----+-----+----+---------------------+---------------------+----------------+

@johnkerl
Copy link
Owner Author

johnkerl commented Feb 19, 2024

I think the issue involves how pprint output does alignment of numeric values:

$ mlr --ojson --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")' then tail -n 2
[
{
  "a": "hat",
  "b": "wye",
  "i": 9,
  "x": 0.03144187646093577,
  "y": 0.7495507603507059,
  "si": 3000000000
},
{
  "a": "pan",
  "b": "wye",
  "i": 10,
  "x": 0.5026260055412137,
  "y": 0.9526183602969864,
  "si": 31622776601
}
]

$ mlr --ocsv --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")' then tail -n 2
a,b,i,x,y,si
hat,wye,9,0.03144187646093577,0.7495507603507059,3000000000
pan,wye,10,0.5026260055412137,0.9526183602969864,31622776601

$ mlr --opprint --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")' then tail -n 2
a   b   i  x                   y                  si
hat wye 9  0.03144187646093577 0.7495507603507059 3000000000
pan wye 10 0.5026260055412137  0.9526183602969864 31622776601

$ mlr --opprint --barred --from test/input/abixy put '$si = fmtnum(int(sqrt($i)*10**NR), "%d")' then tail -n 2
+-----+-----+----+---------------------+--------------------+-------------+
| a   | b   | i  | x                   | y                  | si          |
+-----+-----+----+---------------------+--------------------+-------------+
| hat | wye | 9  | 0.03144187646093577 | 0.7495507603507059 | 3000000000  |
| pan | wye | 10 | 0.5026260055412137  | 0.9526183602969864 | 31622776601 |
+-----+-----+----+---------------------+--------------------+-------------+

@johnkerl johnkerl mentioned this pull request Feb 22, 2024
Open
@chenrui333 chenrui333 mentioned this pull request Mar 20, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@johnkerl @aborruso @lulunac27a