Questions tagged [accuracy]
The accuracy tag has no summary.
148 questions
0
votes
6
answers
125
views
High-accuracy DC current and voltage measurement on PCB
My application requires a high accuracy reading for DC voltage and current. DC voltage max is about 60 V and DC current max is about 140 A.
Currently I am using a 1 mΩ shunt(CSS2H-3920R-1L00F), for ...
0
votes
1
answer
104
views
How is power measured in a domestic smart meter? [duplicate]
I ask this question because my background domestic electricity usage is shown as 200Wh on the display but my measurements show a lower power usage with a power factor of 0.6, so I put a scope on it, ...
1
vote
2
answers
96
views
Are LDO line and load regulation specs useless when the tolerance is so loose?
I'm looking at this ST1L05 voltage regulator for the purpose of an uncertainty/error analysis of a downstream ADC. I'm struck by how loose the output voltage tolerance range is, at least for the 3V3 ...
0
votes
0
answers
52
views
Capacitive encoder accuracy vs repeatability vs ppr
I am trying to understand the properties/limitations of a capacitive encoder (more specifically an AMT10E).
It has a dip switch, by which the ppr can be set between 120 and 5120, which corresponds to ...
3
votes
4
answers
228
views
Measuring wide voltage range via voltage divider "accurately"
I'd like to measure the voltage range 3V .. 30V via a voltage divider. When I use a ratio of 100k to 10k for example and use 1% resistors (taking into account drift due to aging -> 2% or even more) ...
1
vote
1
answer
54
views
Output from MLX90395 unexpectedly slow and inaccurate
I am working with an MLX90395 for a magnetic control project. I started by hooking it up to an Arduino and using the Adafruit MLX90395 library:
https://github.com/adafruit/Adafruit_MLX90395
But there ...
4
votes
7
answers
2k
views
Precise Electronic Measurements
I've been looking into various electronic components and noticed rampant 5%, 10%, and even 20% tolerances and accuracies not only in basic components such as capacitors, inductors, and resistors, but ...
1
vote
0
answers
54
views
"50 Ohm" vs "50 Ohm terminated" in ACV accuracy specs
In the accuracy specifications for an AC voltage measurement with a Keysight 3458A, there is a distinction between source impedance of "50 Ohm" and "50 Ohm terminated" for the &...
0
votes
1
answer
389
views
What is the best and simplest way to generate multiple accurate reference voltages?
I need several reference DC voltages to use them to calibrate hundreds of circuit boards manufactured in our facility.
The voltages to be calibrated are
5V
10V
100V
200V
500V
The accuracy required ...
0
votes
2
answers
217
views
Multimeter Range and Resolution
I'm confused as to how my multimeter takes readings, especially since its not an auto-range multimeter. I have to switch to the desired range to carry out measurements. Unfortunately, the resolution ...
1
vote
3
answers
675
views
Easiest way to get highly-accurate frequency reference?
I just want to check the calibration of my frequency counter, so anything constant under 250 MHz would work(?)
Mains are synced to atomic clocks, to maintain wall clock accuracy, but only ...
1
vote
2
answers
191
views
Understanding accuracy specification for an electronic load box
I am using a BK Precision 8600 electronic load box. I am using it in constant resistance mode (CR mode). I have two resistances I am using, 15Ω and 5Ω, and I am interested in determining the max/min ...
1
vote
3
answers
481
views
Shunt's thermal EMF (Seebeck effect) vs Kelvin connection?
While reading this Vishay document about shunts, I saw that temperature mismatch between the shunt's terminals can introduce a "parasitic" voltage drop which will in turn affects the current ...
0
votes
2
answers
306
views
Highly accurate DIY friendly (cheap) power load
To avoid XY question, I start with my motivation and then my question.
I have an Emporia Vue2 (in-panel) power sensor but it is known not to be very accurate. For me there is a discrepancy between the ...
2
votes
1
answer
109
views
Which INL is better for this ADC?
I am using the following ADC.
I want to know given the figures below, which INL is better?
How do I interpret the x-axis (% of FS)? What is my full-scale (FS) value? My Vin ranges from 10mV to 50mV ...