Skip to main content

Questions tagged [floating-point]

For questions about floating-point, a numerical representation format used in computing to represent and perform arithmetic operations on real numbers.

11 votes
4 answers
4k views

I don't see a reason not to implement such floating-point type, like this (Rust-esque pseudocode): ...
Dannyu NDos's user avatar
  • 1,485
12 votes
0 answers
491 views

By the C standard and the C++ standard: $\log(0+0i) = -\infty+0 i$. $\log(0-0i) = -\infty-0 i$. $\log(-0+0i) = -\infty+\pi i$. $\log(-0-0i) = -\infty-\pi i$. However, if the floating-point complex ...
Dannyu NDos's user avatar
  • 1,485
4 votes
2 answers
420 views

The most common ones are base 2 or base 10 (aka decimal floating point). But can they have any base (like base 3, base 6, base 4 etc.)?
user avatar
7 votes
6 answers
1k views

In some languages, a number literal's type can be inferred from nearby expressions. For example, 1u64 + 2 might give a u64 value ...
rydwolf's user avatar
  • 4,870
14 votes
12 answers
7k views

In common languages, sine produces an error, NaN, or exception when evaluated at infinity. For example, in Python: ...
Corbin's user avatar
  • 901
4 votes
4 answers
530 views

Sometimes, we want to implement floating-point operations on systems which use IEEE 754 representations of values, but don't necessarily support IEEE 754 semantics. For example, we may be using the <...
Corbin's user avatar
  • 901
5 votes
3 answers
418 views

Given a new language, how best should we convert decimal to binary floating point? And please don't just say "use strtod(3)": The venerable C function supports a baroque variety of weird ...
Beth's user avatar
  • 153
12 votes
7 answers
2k views

(Or really just poor conditioning in general.) Recently, there has been some interest in correctly-rounded floating-point libraries (RLIBM and CORE-MATH), which I think is great, but there is one ...
Moonchild's user avatar
  • 1,096
24 votes
5 answers
7k views

Intel created a series of floating-point coprocessors for the x86 called the x87. In addition to supporting IEEE single and double precision numbers, it also supported an 80-bit extended precision ...
Bbrk24's user avatar
  • 9,672
13 votes
8 answers
2k views

So I've seen that many languages return a NaN value when calculating an indeterminate arithmetic expression (e.g. 0/0). Other ...
The Empty String Photographer's user avatar
12 votes
5 answers
5k views

From Wikipedia: In computing, fixed-point is a method of representing fractional (non-integer) numbers by storing a fixed number of digits of their fractional part. ... More generally, the term may ...
justANewbie's user avatar
  • 1,100