Reliability
Reliability is the consistency of measurement: a reliable measurement system gives the same values each time the same behavior is measured.
In behavior analysis, a measurement is reliable when repeated measurement of the same events produces the same result. If an RBT watches the same video of a session twice and gets 12 hits both times, and another trained observer also gets 12, the measurement is consistent. If the counts vary each time, the data are unreliable, and changes on the graph may reflect measurement error rather than changes in behavior.
Reliability is not the same as accuracy. Accuracy means the recorded values match the true values of what happened. A measurement can be consistently wrong: an observer who always misses the same type of response will be reliable but inaccurate. Validity is a third, separate question: whether the right behavior and the right dimension were measured at all.
Unreliable data collection is one of the main risks named in task A.8. Causes include vague definitions, poor training, observer drift, fatigue, distraction, recording from memory after the session, and complicated data systems. RBTs improve reliability by using the definition exactly, recording in the moment, practicing difficult definitions with the supervisor, and taking part honestly in IOA checks.
Example and non-example
| Example | An RBT counts a client's vocal stereotypy from the same 10-minute video on three occasions and gets 31, 31 and 30, showing highly consistent measurement. |
|---|---|
| Non-example | An RBT who records 30 responses when the true number was 45, every time, is consistent but not accurate; high reliability alone does not make data accurate. |
Don't confuse it with
| Validity | Reliability is about getting the same value consistently; validity is about measuring the right behavior and dimension for the question. |
|---|---|
| Interobserver agreement | Reliability is the property of consistent measurement; IOA is a common procedure used to check how believable the data are. |
Test yourself
-
Question 1Which practice is used to check how reliable an RBT's data are?
Show answer
D: A second observer independently records the same session for comparison. Interobserver agreement (IOA) compares the records of two observers who measured the same behavior at the same time without seeing each other's data. Agreement between independent observers is the usual check on how consistent and believable the data are.
- A: A trend shows the direction of the data, not whether the data were recorded consistently.
- B: Recounting tallies only checks arithmetic; it cannot show whether the tallies matched the behavior.
- C: A caregiver's signature confirms the session took place, not that the measurement was reliable.
Task A.8 · 40-hour curriculum section C
-
Question 2An RBT is busy during a 3-hour home session and waits until the session ends to write down how many times the client hit, estimating the number from memory. What is the main risk of this practice?
Show answer
C: The count may not match what happened, leading to wrong decisions. Counts reconstructed from memory are estimates, and estimates are easily too high or too low. The supervisor relies on these numbers to decide whether the plan is working, so inaccurate counts can lead to the wrong decision. Behavior should be recorded as it occurs, following the data procedure in the plan.
- A: Nothing about late recording by itself would make the hitting increase.
- B: A rate can still be calculated from any count and time; the problem is that the count may be wrong.
- D: Recording late does not turn a count into an interval or time-sampling procedure; it simply makes the count unreliable.
Task A.8 · 40-hour curriculum section C · RBT Ethics Code 2.02
-
Question 3Two RBTs record a client's hitting on alternate days. One counts every hit in a burst separately; the other counts a burst of hits as one occurrence. The graph shows much higher counts on the first RBT's days. What is the best interpretation and response?
Show answer
A: The RBTs may be recording differently and should ask the supervisor to clarify the definition. When people record the same behavior with different rules, the data change with the recorder rather than the client. The fix is a shared, clear operational definition from the supervisor, used by everyone.
- B: The difference can be explained by the recording rules alone, and an RBT does not change the plan on their own.
- C: Changing the recording rule without telling the supervisor leaves a hidden break in the data and bypasses the person responsible for the definition.
- D: Averaging two records made with different rules does not produce an accurate count.
Task A.8 · 40-hour curriculum section C
Answer every question to see your score.
On the RBT exam
This term belongs to task A.8 Unreliable data and poor procedural fidelity in content area A, Data Collection and Graphing, which is 13 of the 75 scored questions.