Formula:Precision
4
Documentatio formulae[ostende] [recense] [historiam inspice] [purga]The Template:Precision determines the precision (as a count of decimal digits) for any amount, large or negative, using a fast algorithm. It can also handle a trailing decimal point (such as "15." or "-41.") or trailing zeroes (such as "15.34000" having precision as 5 decimal digits). For fractional input it returns the base ten logarithm of the numerator.
Examples
recensere{{Precision|1111.123456789}} |
9 |
{{Precision|1111.12345678}} |
8 |
{{Precision|1111.1234567}} |
7 |
{{Precision|1111.123456}} |
6 |
{{Precision|1111.12345}} |
5 |
{{Precision|1111111111.12345678}} |
8 |
{{Precision|1111111111.1234567}} |
7 |
{{Precision|1111111111.123456}} |
6 |
{{Precision|1111111111.12345}} |
5 |
{{Precision|1111111111.1234}} |
4 |
{{Precision|1111111111.123}} |
3 |
{{Precision|1111111111.12}} |
2 |
{{Precision|1111111111.1}} |
1 |
{{Precision|1111111111.10}} |
2 |
{{Precision|1111111111.100}} |
3 |
{{Precision|1111111111.1000}} |
4 |
{{Precision|1111111111.10000}} |
5 |
{{Precision|1111111111}} |
0 |
{{Precision|1111111110}} |
-1 |
{{Precision|1111111100}} |
-2 |
{{Precision|1111111000}} |
-3 |
{{Precision|1111110000}} |
-4 |
{{Precision|1111100000}} |
-5 |
{{Precision|1111000000}} |
-6 |
{{Precision|1110000000}} |
-7 |
{{Precision|1100000000}} |
-8 |
{{Precision|0}} |
0 |
{{Precision|1}} |
0 |
{{Precision|22.45}} |
2 |
{{Precision|22.12345}} |
5 |
{{Precision|22}} |
0 |
{{Precision|22000}} |
-3 |
{{Precision|-15.275}} |
3 |
{{Precision|-15.2500}} |
4 |
{{Precision|23000222000111.432}} |
3 |
{{Precision|-15.123}} |
3 |
{{Precision|0.09}} |
2 |
{{Precision|0.88}} |
2 |
{{Precision|880000}} |
-4 |
{{Precision|90000000}} |
-7 |
Known bugs
recensere- For numbers in scientific notation, the precision is typically returned as too low by 1 decimal place. Example: {{precision |7.1234E+06}} → -2 (should be precision as 4 decimal digits, not 3).
- Large numbers are limited to 11 trailing zeroes, so even larger numbers still report precision as being -11, such as 9 trillion: {{precision|9000000000000}} → -12 (should be: -12).
Technical notes
recensere- NOTE A1: This template determines the precision of decimals by counting the length of the numeric string (in a #switch comparing lengths of padded strings), then subtracting integer length, minus the decimal point, and minus 1 if negative. For integers, 1 place is subtracted for each trailing 0 on the integer. For fractions, any prior count is cleared x 0, then returns the base ten logarithm of denominator: (..prior...)*0 + (ln denom / ln 10).
- NOTE D2: The check, for whole integers, compares the amount versus appending "0" at the end: when the amount is a decimal, then the value is unchanged by appending 0 at the end: so 5.23 = 5.230 is true, whereas for whole integers, it would be: 5 = 50 as false, due to values becoming n*10 for integer n. So, for integer n, the check rejects: n = n0 as false; hence n is integer.
- NOTE M3: The magnitude of the integer portion is calculated by logarithm of the floor of absolute value (divided by natural logarithm of 10 to adjust for e=2.71828*), as: ln (floor( abs() )+0.99 )/ln10 Function floor(x) trims the decimal part, to leave the whole count: 0-9 yield 0, 10-19 as 1, 1000-1999 as 3. The abs(x) avoids floor of negatives, floor(-0.1)= -1, hence using abs(x) ensures -0.1 floors to 0 not -1. Near zero, the +0.99 avoids invalid log of 0, but does not round-up any decimals, already floored as nnn.00. Complexity is 6 operations: floor of abs( {1} ) +0.99 then log10x (lnx ÷ ln10), then floor that logarithm ratio. Decimals -1 < x < 1 yield -1, avoiding log 0.001 = -3.
- NOTE N4: Nesting of if-else and nested templates is kept to a minimum, due to the MediaWiki 1.6 limit of 40 levels of if-logic for all nested templates used together. Template {ordomag} was omitted to avoid 2 more levels of nested templates. Template {Precision} had 8 levels, and this template was trimmed to only 5 levels.
- NOTE S5: The #switch is run with "x" prepended in front of the amount, otherwise a #switch will compare as numeric where "2" would match "2.0" even though "2" is length 1 so "x2" no longer matches with "x2.0" as non-numeric. The #switch will exit on the first match, so smaller lengths are compared first, to avoid extra comparisons for more rare, longer numeric strings up to 41 long.
- NOTE W6: The check for integers with whole end-zeroes uses typical n=n/10*10, for each power of 10, where whole millions match: {{#ifexpr: {1}=floor( {1}/1E6 )*1E6| }} Previously, {Precision} had tried to use "round" to detect end-zeroes but "round" loses precision at -5, so, n00000 round -5 differs from n00000 slightly, and comparisons to exact rounded amounts failed to match some numbers when 6 or more zeroes "n000000".
- NOTE Z7: The check on zero for any .00000 compares adding 1 to the amount, versus appending "1" at the end: if the amount is a decimal, then adding 1 will be larger than appending 1 at the end: 0.00 + 1 > 0.001, whereas for whole zero, it would be: 0+1 > 01 as false, due to the value being the same. So, for integer 0, the check rejects: 0+1 > 01 as false; hence whole 0 is integer.
Vide etiam
recensereDocumentatio pristina transclusa est ex Formula:Precision/doc. (recense | historiam inspice)
Quaesumus auctores experturos esse in harenarii (crea | effinge) ac periclitationum (crea) subpaginis huius formulae.
Lubet categorias et nexus in /doc subpaginam addere. Vide etiam subpaginas huius formulae.