All Questions
2 questions
0
votes
0
answers
25
views
Deviation caused by half() in ptyroch
I have met a question that the value of a tensor is 6.3982e-2 in float32. After I changed it to float16 using half() function, it became 6.3965e-2. Will there be a method to convert tensor without ...
- 13
17
votes
1
answer
28k
views
How to select half precision (BFLOAT16 vs FLOAT16) for your trained model?
how will you decide what precision works best for your inference model? Both BF16 and F16 takes two bytes but they use different number of bits for fraction and exponent.
Range will be different but I ...
- 1,386