JOURNAL OF APPLIED BEHAVIOR ANALYSIS
2014, 47, 70–82
NUMBER
1 (SPRING)
PARAMETERS OF REINFORCEMENT AND RESPONSE-CLASS HIERARCHIES GRACIE A. BEAVERS GEORGIA STATE UNIVERSITY
BRIAN A. IWATA UNIVERSITY OF FLORIDA
AND
MEAGAN K. GREGORY FLORIDA INSTITUTE OF TECHNOLOGY
Shabani, Carr, and Petursdottir (2009) examined the effects of a response–response relation (effort) on the development of a response-class hierarchy using a laboratory model. Response–reinforcer relations may have similar influences. Using a similar translational approach, we examined the effects of reinforcer rate, quality, delay, and magnitude in a series of separate experiments conducted with 8 individuals with intellectual disabilities. Response-class hierarchies emerged along the dimension of rate for 3 of 6 subjects, quality for 5 of 5 subjects, delay for 2 of 8 subjects, and magnitude for 5 of 6 subjects. Key words: reinforcer dimensions, response class
in the frequency of the other responses are observed (Harding et al., 2001; Lalli, Mace, Wohn, & Livezey, 1995; Magee & Ellis, 2000; Shukla-Mehta & Albin, 2003). Results from this line of research have raised several questions about treatment of problem behavior, and, in particular, the potential for increasing more severe problem behavior when less severe problem behavior is extinguished. Noting the difficulty of conducting studies on response-class hierarchies with actual problem behavior, Shabani, Carr, and Petursdottir (2009) illustrated a laboratory model of a response-class hierarchy along the dimension of response effort. In Study 1, they first trained four children to press each of three buttons (low, medium, and high effort) separately on fixed-ratio (FR) 1 schedules of reinforcement. They then reinforced responses on all three buttons on a concurrent FR 1 schedule and observed that subjects responded primarily on the low-effort button. When responses on the low-effort button subsequently were extinguished, subjects responded mainly on
A response class consists of behaviors that are topographically different but functionally similar in that they are maintained by the same reinforcing consequences, and a hierarchical response class is one in which responses tend to occur in a particular order (Baer, 1982). The development of response-class hierarchies may be influenced by response–response relations, such as effort, or response–reinforcer relations, including reinforcer rate, quality, delay, or magnitude (Mace, 1994; Mendres & Borrero, 2010). Applied research on response-class hierarchies has proceeded from anecdotal observation, followed by a functional analysis whose results identified a common maintaining variable for all of the responses, and a final analysis in which one member of the class is extinguished while changes We thank Tara Fahmie and Amanda Rone for their assistance. Address correspondence to Brian A. Iwata, Department of Psychology, Room 114 Psychology Building, University of Florida, Gainesville, Florida 32611 (e-mail: [email protected]). doi: 10.1002/jaba.102
70
RESPONSE CLASS the medium-effort button. When responses on all buttons again were reinforced, subjects’ low-effort responding reemerged while medium- and higheffort responding decreased. In Study 2, Shabani et al. conducted a further analysis by extinguishing responses on each of the three buttons. When responses on the low-effort button were placed on extinction, subjects usually responded on the medium-effort button first. When responses on the medium-effort button were placed on extinction, subjects usually pressed the low-effort button first. When responses on both the lowand medium-effort buttons were placed on extinction, the subjects usually responded on the high-effort button. These results supported the interpretation that the three button presses had become members of the same response class and that the responses were hierarchically organized along the dimension of response effort. Mendres and Borrero (2010) replicated and extended the results of Shabani et al. with 11 college students. Mendres and Borrero operationalized effort as differences in reinforcement schedule requirements and showed that responseclass hierarchies developed under contingencies of both positive and negative reinforcement. In addition to response effort, several parametric features of reinforcement contingencies that influence operant behavior in general also may play a role in the development of response-class hierarchies. The purpose of this study was to extend previous research by examining the influences of reinforcer rate, quality, delay, and magnitude on the formation of response-class hierarchies. Although the arrangement used by Mendres and Borrero (2010) was designed to examine the effects of response effort, their manipulation of reinforcement schedules also inherently involved the dimension of reinforcer rate. We minimized differences in schedule requirements, thereby separating the effects of rate and effort. Another difference between the Shabani et al. (2009) and Mendres and Borrero studies and ours was the conditions under which responding was established initially. In their
71
studies, each response first was established under a separate arrangement. These individual histories of reinforcement may have facilitated the rapid appearance of a hierarchy later when all responses were reinforced concurrently. We wanted to see whether hierarchies would emerge naturally as a result of differential but concurrent reinforcement from the outset of training, which may reflect the way in which many response-class hierarchies actually develop.
METHOD Subjects and Setting Eight individuals, all of whom had been diagnosed with an intellectual disability, participated in one or more manipulations. Mack (40 years old), James (33 years old), Jimmy (49 years old), Rita (69 years old), Nancy (40 years old), Angela (46 years old), and Hank (49 years old) were recruited from a day-training facility for adults with intellectual disabilities; Grayson (18 years old) attended classes in a special education school. All subjects were capable of following simple instructions and performing the experimental task, and none had been reported to engage in significant problem behavior. Sessions were conducted in a quiet room at the day program or the school that contained a table, chairs, and materials needed to conduct sessions (see below). One to five sessions were conducted per day, usually 5 days per week. Response Measurement and Reliability The target response was touching one of three cards that were located on a table in front of the subject. Observers used handheld computers to collect data, which were summarized as the percentage of trials on which responses occurred on each card. An independent observer collected data during a minimum of 25% of sessions for each subject in every experimental condition. Although data were summarized as a percentage of trials, data were collected using a continuous
72
GRACIE A. BEAVERS et al.
recording procedure. Thus, a proportional reliability calculation was used in which each session was divided into continuous 10-s intervals, and observers’ records were compared on an interval-by-interval basis. The smaller number of responses was divided by the larger in each interval, and these fractions were averaged across the session. Mean interobserver agreement scores across subjects in each manipulation were as follows: reinforcer rate, 97% (range, 95% to 98%); reinforcer quality, 97% (range, 95% to 98%); reinforcer delay, 97% (range, 96% to 98%); and reinforcer magnitude, 96% (range, 92% to 98%). General Procedure A reversal design was used to evaluate the influence of each parameter of reinforcement on the development of a response-class hierarchy. In each manipulation, one parameter of reinforcement varied while the others were held constant. A session consisted of 20 trials. Each trial began with the presentation of three cards and ended when a response was made on one of the cards or when 30 s elapsed with no response to any card. The position of the cards was rotated after every trial. The intertrial interval was held constant at 30 s to equate trial duration across all conditions. Before each session, the experimenter prompted the subject to touch each card and delivered the consequence associated with that card touch in the current condition. This presession forced exposure was conducted to increase the salience of the consequences associated with each card touch and to decrease the likelihood of carryover across conditions. Preference Assessment Before each manipulation, a paired-stimulus preference assessment (Fisher et al., 1992) was conducted with nine edible items that the subject had sampled beforehand. Items were selected by the experimenter from an array of available supplies at each site and included a
variety of tastes and textures. On each trial, the experimenter presented two items on separate plates in front of the subject and asked the subject to select one of the items. The subject was allowed to consume the selected item, and the other item was removed. Trials continued until each item had been paired once with every other item, and results were summarized as the percentage of trials on which an item was selected. Items selected on 80% or more of trials were considered highly preferred and were used as reinforcers in the rate, delay, and magnitude manipulations. Additional criteria were used to select preferred stimuli in the reinforcer quality manipulation (see “Reinforcer Quality” below). Reinforcer Rate Three different colored cards were placed on the table in front of the subject; each card was paired with a different FR schedule of reinforcement (FR 1, FR 5, or FR 10). Each card also displayed the number of responses (1, 5, or 10) required to obtain reinforcement. A modified card touch was used with individuals who demonstrated difficulty in discriminating the contingencies associated with each card: Velcro tabs corresponding to the schedule requirements were affixed to each card, and subjects were required to remove the tabs to complete the schedule. If the subject touched the FR 5 or FR 10 card, the other cards were removed, and the subject had 30 s to complete the ratio before the next trial began. Contingent on ratio completion, the therapist placed one piece of the highly preferred edible item identified in the preference assessment on a plate in front of the subject (except when the response associated with that schedule was placed on extinction). The subject could consume the earned edible items at any time during the session. Baseline. During baseline, no programmed consequences were delivered for touching any of the cards or engaging in any other response. Reinforcement (all). In this condition, responses on all cards were reinforced (provided
RESPONSE CLASS the schedule requirements were met before 30 s elapsed). Reinforcement (FR 5/FR 10). In this condition, only responses on the cards associated with the FR 5 and FR 10 schedules were reinforced; responses on the FR 1 card were placed on extinction. Reinforcement (FR 10). In this condition, only responses on the FR 10 card were reinforced; responses on the FR 1 and FR 5 cards were placed on extinction. Additional training conditions. The following conditions were conducted only if a responseclass hierarchy did not emerge in the initial reinforcement (all) condition: (a) In the reinforcement (FR 5) condition, only responses on the FR 5 card were reinforced; responses on the FR 1 and FR 10 cards were placed on extinction. (b) In the reinforcement (FR 1) condition, only responses on the FR 1 card were reinforced; responses on the FR 5 and FR 10 cards were placed on extinction. Reinforcer Quality In this manipulation, three different colored cards were associated with high-, moderate-, and low-preference reinforcers, respectively. Highpreference (HP) items were those selected on 90% or more of trials in the preference assessment, moderate-preference (MP) items were those selected on between 25% and 50% of trials, and low-preference (LP) items were those either never selected or selected the least. A single-stimulus preference assessment was conducted with items from the LP group to ensure that these items had some reinforcing value. Each LP item was presented singly on five separate trials. An item that was consumed on all five trials was included as the LP item in experimental sessions. Before the session, HP and MP items, as well as MP and LP items, were paired at least 10 but no more than 20 times to ensure that the subject’s preference hierarchy was maintained in the subsequent sessions. First, HP and MP items were paired: If the subject selected the HP item
73
on the first 10 presession trials, no other HP/MP trials were conducted, and the HP item remained HP for the subsequent session. If the subject’s selection varied during the first 10 trials, 10 additional trials (20 total) were conducted, and the item selected most frequently was designated HP for the subsequent session. After the HP/MP presession trials, MP and LP items were paired using the same procedures as described for the HP/MP presession trials. After the preference hierarchy had been established, the experimental session began. During the session, contingent on a card touch, the therapist placed one piece of the corresponding edible item on a plate in front of the subject (except when the response emitted was placed on extinction). The subject could consume the earned edible items at any time during the session. Baseline. In baseline, no programmed consequences were delivered for touching any of the cards or engaging in any other response. Reinforcement (all). In this condition, touches on all cards were reinforced; that is, a touch on the HP card was followed by delivery of the HP edible item, and so on. Reinforcement (MP/LP). In this condition, only responses on the MP and LP cards were reinforced; responses on the HP card were placed on extinction. Reinforcement (LP). In this condition, only responses on the LP card were reinforced; responses on the HP and MP cards were placed on extinction. Reinforcer Delay In this manipulation, three different-colored cards were associated with three delays to reinforcement: 1 s, 10 s, and 30 s. Digital timers that displayed the corresponding delays were placed behind the 10-s and 30-s cards. Contingent on a card touch, the therapist either immediately placed one piece of the edible item on a plate in front of the subject (1-s delay) or started the timer with the delay corresponding to the card (10-s or 30-s delay) and delivered the reinforcer when the time elapsed
74
GRACIE A. BEAVERS et al.
(except when the response emitted was placed on extinction). The subject could consume the earned edible items at any time during the session. Baseline. In baseline, no programmed consequences were delivered for touching any of the cards or engaging in any other response. Reinforcement (all). In this condition, touches on all cards were reinforced; responses on the 1-s card resulted in a 1-s delay to delivery of reinforcement, and so on. Reinforcement (10 s/30 s). In this condition, only responses on the 10-s and 30-s delay cards were reinforced; responses on the 1-s delay card were placed on extinction. Reinforcement (30 s). In this condition, only responses on the 30-s delay card were reinforced, and responses on the 1-s and 10-s delay cards were placed on extinction. Additional training conditions. The following conditions were conducted only if a responseclass hierarchy did not emerge in the initial reinforcement (all) condition: (a) Only responses on the 10-s delay card were reinforced, and responses on the 1-s and 30-s delay cards were placed on extinction, or (b) only responses on the 1-s delay card were reinforced, and responses on the 10-s and 30-s delay cards were placed on extinction. Reinforcer Magnitude In this manipulation, three different-colored cards were associated with three magnitudes of reinforcement: one, two, and five pieces of an edible item. Each card also displayed the number (1, 2, 5) corresponding with the number of pieces of reinforcement. One subject (Grayson) verbally stated a preference for the number 2; thus, for him, small, medium, and large pieces of one edible item were used (the small, medium, and large pieces were one quarter, one half, and one whole edible item, respectively). Contingent on a card touch, the therapist placed the corresponding number of pieces of the edible item on a plate in front of the subject (except when the response emitted was placed on extinction). The subject
could consume the earned edible items at any time during the session. Baseline. In baseline, no programmed consequences were delivered for touching any of the cards or engaging in any other response. Reinforcement (all). In this condition, touches on all cards were reinforced with the corresponding number of pieces (or sizes) of the edible item. Reinforcement (M2/M1). In this condition, only responses on the M2 and M1 (medium and small) cards were reinforced; responses on the M5 (large) card were placed on extinction. Reinforcement (M1). In this condition, only responses on the M1 (small) card were reinforced, and responses on the M2 and M5 (medium and large) cards were placed on extinction. Additional training conditions. The following conditions were conducted only if a responseclass hierarchy did not emerge in the initial reinforcement (all) condition: (a) Only responses on the M2 (medium) card were reinforced, and responses on the M1 and M5 (small and large) cards were placed on extinction, or (b) only responses on the M5 (large) card were reinforced, and responses on the M1 and M2 (small and medium) cards were placed on extinction.
RESULTS Results for each subject were considered indicative of the development of a response-class hierarchy if (a) in the reinforcement (all) condition, the majority of responses occurred on the card associated with the most favorable parameter of reinforcement (i.e., the densest rate, the highest quality, the shortest delay, or the largest magnitude), and (b) when responding on one or more of the cards was placed on extinction, the majority of responses occurred on the card associated with the next most favorable parameter of reinforcement. In other words, relative response probabilities observed over the course of experimental conditions were used as the basis
RESPONSE CLASS for determining that a hierarchy of responses had developed (Catania, 2007). Figure 1 shows representative results for individuals for whom a response-class hierarchy emerged as predicted by the parameters of reinforcement in each manipulation: Jimmy in the rate manipulation, James in the quality manipulation, Mack in the delay manipulation, and Grayson in the magnitude manipulation. In baseline, all subjects exhibited zero or near-zero responding on all cards, with the exception of Grayson, who responded on all cards before ceasing altogether. When reinforcement was delivered for responses on all cards in the reinforcement (all) condition, all subjects allocated the majority of their responding to the card associated with the most favorable parameter of reinforcement. Mack never responded on the other cards, whereas James showed some initial responding on the other cards that decreased across sessions, and Jimmy and Grayson occasionally responded on the other cards throughout the condition. When responding on the card associated with the most favorable parameter of reinforcement was extinguished, and only responses on the other cards were reinforced, all subjects shifted their responding to the card associated with the next most favorable parameter of reinforcement. When responses on all cards were again reinforced, all subjects again responded primarily on the card associated with the most favorable parameter of reinforcement. When reinforcement was only delivered following responses on the card associated with the least favorable parameter of reinforcement and responses on the other cards were extinguished, all subjects responded mainly on the card associated with reinforcement. In the final reinforcement (all) condition, all subjects again allocated the majority of their responses to the card associated with the most favorable parameter of reinforcement. Other subjects for whom a response-class hierarchy emerged as predicted by the parameters of reinforcement in each manipulation showed
75
results closely resembling those in Figure 1 with one exception: During the reinforcement (LP) condition of the quality manipulation, three subjects’ responding varied from the pattern exhibited by James (Figure 1). Grayson initially responded on the card associated with the LP item, but his responding decreased to zero on all cards as the condition continued. Nancy and Rita never responded on the card associated with the LP item at any point during the reinforcement (LP) condition. Additional evidence of hierarchical development was obtained from analyses of subjects’ responding on the first trial of the first session of each condition after baseline. That is, the larger the percentage of trials on which subjects immediately switched to the card as predicted by a given parameter of reinforcement, the more likely a hierarchy had formed. The percentage of trials on which an immediate switch to the predicted card was observed for all subjects for whom a response-class hierarchy emerged in the rate, quality, delay, and magnitude manipulations was 66.7%, 80%, 40%, and 64%, respectively. Thus, the strongest additional evidence of hierarchical development was associated with the quality manipulation. Figure 2 shows typical results for individuals for whom a response-class hierarchy did not emerge as predicted by the parameters of reinforcement: Angela in the rate manipulation, Hank in the delay manipulation, and Rita in the magnitude manipulation. In baseline, two different patterns were observed: Angela and Hank did not respond on any of the cards, whereas Rita responded evenly on all cards despite the absence of all programmed consequences. In the initial reinforcement (all) condition, all three subjects allocated responding equally to all cards. Because these subjects did not show the emergence of a response-class hierarchy during the initial reinforcement (all) condition, additional training conditions were conducted to ensure that subjects discriminated the contingencies associated with each card. When each parameter of
GRACIE A. BEAVERS et al.
76 Baseline
Sr+(All)
Sr+(FR 5/FR 10)
Sr+(All)
Sr+(FR 10)
Sr+(All)
100 FR 5
FR 1
80 60
FR 10 JIMMY (RATE)
40 20 0
Baseline
Sr+(All)
Sr+(MP/LP)
Sr+(All)
Sr+(LP)
Sr+(All)
100 80
HP
60
CARD TOUCH (% TRIALS)
40
JAMES (QUALITY)
LP MP
20 0
100
Baseline
Sr+(All)
Sr+(10 s/30 s)
1s
80
Sr+(All)
Sr+(30 s)
Sr+(All)
10 s
60 MACK (DELAY)
30 s
40 20 0
100
Baseline
Sr+(All)
80
Sr+(Med/Small)
Sr+(All)
Sr+(Small)
Sr+(All)
Large Med
60 GRAYSON (MAGNITUDE)
40 Small 20 0 5
10
15
20
25
30
35
40
SESSIONS Figure 1. Representative data for individuals who showed emergence of response-class hierarchies as predicted by the rate, quality, delay, and magnitude manipulations. Srþ ¼ reinforcement.
reinforcement was isolated in a separate condition (three values for rate, delay, and magnitude), subjects responded according to the prevailing contingency. That is, when responding on only one card at a time was reinforced (responding on the other cards was extinguished), all subjects
showed clear response differentiation. However, when reinforcement (all) was reimplemented, these subjects again responded indifferently. Other subjects for whom a response-class hierarchy did not emerge as predicted by the parameters of reinforcement in each manipulation
RESPONSE CLASS 100
Baseline
Sr+(All)
Sr+(FR 10)
Sr+(FR 5)
77 Sr+(FR 1)
Sr+(All)
80 60
FR 5
40 ANGELA (RATE)
20 FR 10 FR 1
CARD TOUCH (% TRIALS)
0 Baseline
Sr+(All)
Sr+(1 s)
Sr+(10 s)
Sr+(30 s)
Sr+(All)
100 80 10 s
1s
60 40 20
HANK (DELAY)
30 s 0
100
Baseline
Sr+(All)
Sr+(M 1)
Sr+(M 2)
Sr+(M 5)
Sr+(All)
80 60
M2
40 RITA (MAGNITUDE)
20 M5 M1 0 5
10
15
20
25
30
35
40
SESSIONS Figure 2. Representative data for individuals who failed to develop response-class hierarchies as predicted by the rate, delay, and magnitude manipulations. Srþ ¼ reinforcement.
showed results closely resembling those in Figure 2 with two exceptions, both seen in Figure 3. Nancy showed emergence of a hierarchy but not of the type predicted. In the initial reinforcement (all) condition of the rate manipulation, Nancy responded more frequently on the less favorable FR 5 card than on the FR 1 card. When responses on each card (FR 1, FR 5, FR 10) were isolated in separate reinforcement conditions, Nancy’s responding showed orderly effects. When the reinforcement (all) condition was reinstated,
however, she again allocated more responses to the FR 5 card. When responding only on the FR 1 and FR 10 cards was reinforced, Nancy allocated the majority of her responding to the FR 1 card. In the final reinforcement (all) condition, Nancy again responded primarily on the FR 5 card. Grayson initially showed the typical emergence of a response-class hierarchy in the delay manipulation. In the initial reinforcement (all) condition, he responded primarily on the card associated with the 1-s delay. When responses
GRACIE A. BEAVERS et al.
78
100
Baseline
Sr+ (All)
Sr+ (FR 1)
Sr+ Sr+ (FR 5) (FR 10)
Sr+ (All)
Sr+ (FR 1/FR 10)
Sr+ (All) NANCY (RATE)
80 FR 5
CARD TOUCH (% TRIALS)
60
FR 1
40
FR 10
20 0 Sr+ (All)
BL
Sr+ Sr+ Sr+ Sr+ (10 s/30 s) (All) (1 s/30 s) (1 s)
Sr+ (All)
100
GRAYSON (DELAY)
80 60 40
1s 10 s 30 s
20 0 5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
SESSIONS Figure 3. Data showing emergence of atypical response-class hierarchies. Srþ ¼ reinforcement.
only on the 10-s and 30-s delay cards were reinforced, Grayson responded mainly on the 10s delay card. However, when we reinstated the reinforcement (all) condition, Grayson’s responding on the 1-s delay card did not increase as it had in the prior reinforcement (all) condition; instead, he continued to respond primarily (and later exclusively) on the 10-s delay card. The next condition revealed an even more unusual pattern of results. When responses only on the 1-s and 30-s delay cards were reinforced, Grayson responded mainly on the 30-s delay card. We then reinforced responses only on the 1-s delay card, and differential responding was observed. In the final reinforcement (all) condition, Grayson again responded primarily on the 10-s delay card. Table 1 summarizes the results for all subjects in all manipulations, the order in which subjects were exposed to each manipulation, and the total number of sessions conducted within each manipulation. Six subjects participated in the reinforcer rate manipulation (M assessment duration ¼ 45 sessions). James, Jimmy, and Mack
showed development of a response-class hierarchy, whereas Angela, Nancy, and Rita did not. Five subjects (Grayson, James, Mack, Nancy, and Rita) participated in the quality manipulation (M assessment duration ¼ 43 sessions), and all five subjects’ responding was indicative of the development of a response-class hierarchy. All subjects participated in the delay manipulation (M assessment duration ¼ 42 sessions). James and Mack showed development of a response-class hierarchy; Angela, Grayson, Hank, Jimmy, Nancy, and Rita did not. Of the six subjects who participated in the magnitude manipulation (M assessment duration ¼ 42 sessions), Grayson, James, Jimmy, Mack, and Nancy developed a response-class hierarchy, but Rita did not. DISCUSSION In a series of separate manipulations, we determined whether response-class hierarchies emerged naturally given disparities in the rate, quality, delay, and magnitude of reinforcement
RESPONSE CLASS Table 1 Summary of Results Subject
Rate
Quality
Delay
Magnitude
Angela Grayson Hank James Jimmy Mack Nancy Rita Total
N (1) 35 — — Y (1) 38 Y (1) 35 Y (1) 33 N (1) 92 N (1) 35 3/6
— Y (1) 47 — Y (3) 38 — Y (2) 36 Y (2) 37 Y (2) 59 5/5
N (2) 31 N (2) 52 N (1) 37 Y (2) 47 N (3) 48 Y (3) 35 N (4) 48 N (4) 37 2/8
— Y (3) 38 — Y (4) 34 Y (2) 61 Y (4) 30 Y (3) 55 N (3) 32 5/6
Note. Y indicates development of a response-class hierarchy, N indicates either no or atypical development of a response-class hierarchy, and a dash indicates the specified manipulation was not conducted with that subject. Numbers in parentheses indicate the order in which the subject was exposed to a manipulation, and the numbers after the parentheses indicate the number of sessions conducted in each manipulation.
for different responses. Response-class hierarchies emerged along the dimension of rate for three of six subjects, quality for five of five subjects, delay for two of eight subjects, and magnitude for five of six subjects. Ideally, each parameter would have been evaluated with each subject; unfortunately, setting constraints prevented this. Nevertheless, these results extend the research of Shabani et al. (2009) and Mendres and Borrero (2010) by demonstrating that response-class hierarchies were sensitive to each of the parameters of reinforcement evaluated for at least some subjects and that, for these subjects, hierarchical responding emerged as a function of parametric differences in reinforcement from the outset of training. The most consistent results were seen with the quality and magnitude manipulations. The robust effects of reinforcer quality on choice allocation have been demonstrated previously by Neef and colleagues in a series of studies that showed that reinforcer quality may override the influence of both reinforcer rate and delay for some individuals (Neef, Bicard, & Endo, 2001; Neef & Lutz, 2001; Neef et al., 2005; Neef,
79
Mace, & Shade, 1993; Neef, Mace, Shea, & Shade, 1992; Neef, Shade, & Miller, 1994). Another possible explanation was the salience of the response–reinforcer relations associated with each card in the manipulations: Although we attempted to enhance the salience of the contingencies in the rate and delay manipulations (i.e., different numbers of Velcro tabs for some subjects in the rate manipulation; timers facing all subjects in the delay manipulation), it is possible that the visual presence of different edible items in the quality manipulation and different numbers of edible items in the magnitude manipulation were stronger discriminative stimuli for these contingencies. Unlike Mendres and Borrero (2010), we did not observe consistent development of responseclass hierarchies in our rate manipulation. This discrepancy might be due to our use of a smaller range of schedules (FR 1, 5, and 10) than those used by Mendres and Borrero (FR 5, 15, and 25). We selected smaller ratios to minimize differences in reinforcer delay across ratios; however, it should be noted that even at smaller ratios, an increase in reinforcer delay is inherent to a larger schedule. Other differences across studies, such as the populations from which subjects were drawn, or the fact that we initially exposed subjects to all schedules concurrently, also may have been responsible for differences in results. Future research could determine whether these variables have any systematic influence on responding. Although hierarchies emerged in 15 of 25 cases in our study, they did not emerge (at least, based on the most favorable parameters of reinforcement) in 10 cases, despite histories of exposure to separate training conditions. That is, if a response-class hierarchy did not emerge in the initial concurrent reinforcement condition, additional training procedures had no effect on the future development of a response-class hierarchy. In spite of relatively clear differences in conditions of reinforcement, some subjects responded indifferently when all responses were
80
GRACIE A. BEAVERS et al.
reinforced or seemed to develop unusual preferences (e.g., Nancy preferred the FR 5 schedule over the FR 1 schedule) based on unknown factors. A similar situation might exist when treatment for problem behavior involves delivery of reinforcement for alternative behavior under an arrangement that is more favorable than that for problem behavior, but problem behavior persists nevertheless (e.g., DeLeon, Fisher, Herman, & Crosland, 2000). Thus, although more favorable conditions of reinforcement often may be sufficient to alter response allocation, other factors may determine the conditions under which a response-class hierarchy emerges and is maintained. One subject (Grayson) admitted a preference for a specific number, which, if unknown, may have affected the development of a response-class hierarchy in the magnitude manipulation. Unfortunately, it was not always possible to identify through client verbal report what extraneous variable may have influenced some subjects’ atypical responding. If the influential variable can be identified, then manipulations can be made to restore optimal responding. Results of studies such as the present one have several implications for practice, especially when attempting to extinguish problem behavior, because more severe problem behavior may emerge. For example, Lalli et al. (1995), after conducting a functional analysis of a child’s screaming, aggression, and self-injury and finding that all responses were maintained by negative reinforcement (escape from demands), systematically placed each response on extinction. When screaming was extinguished, the latency to aggression and self-injury decreased. However, when screaming was reinforced, no instances of self-injury and very few instances of aggression occurred. Thus, if a client is referred for treatment of a particular topography of problem behavior, but caregivers report a history of occasional occurrences of other topographies of problem behavior, care should be taken to ensure appropriate treatment strategies are in place in
the event that other, potentially more severe, topographies of problem behavior emerge over the course of treatment. Response–response and response–reinforcer relations also may affect the conditions under which replacement behavior is established or maintained. For example, Grow, Kelley, Roane, and Shillingsburg (2008) investigated the emergence of appropriate responses during extinction of problem behavior. Following a baseline condition in which inappropriate behavior was reinforced and no programmed consequences were delivered for other responses, the authors extinguished inappropriate behavior and delivered the consequences that maintained inappropriate behavior contingent on another response previously emitted by the subject. Results showed that these untrained alternative responses occurred at higher rates when problem behavior was placed on extinction, despite the lack of explicit training of the alternative response. The authors suggested that both the inappropriate and appropriate responses were members of the same response-class hierarchy, and that the inappropriate response occurred most frequently in the natural environment because of some response- or reinforcer-related variable. One limitation of laboratory analyses of response classes is that results may not accurately reflect what happens in the real world. In particular, our use of a trial-based arrangement was obviously different from what often occurs in real-world settings. Future research might attempt to replicate these results using free-operant procedures similar to those of Shabani et al. (2009) and Mendres and Borrero (2010). Because we isolated variables singly to identify the influence of each, we cannot speak to the effects of combinations of variables, which may covary simultaneously in the natural environment. Future research may examine the influence of combinations of variables on the development of response-class hierarchies, because this information may be useful in selecting optimal treatments for problem behavior. Results of
RESPONSE CLASS studies on response-class hierarchies also may have applications for other clinical populations. For example, Silverman, Chutuape, Bigelow, and Stitzer (1999) showed that larger magnitudes of voucher reinforcement were more effective than smaller magnitudes in reducing the frequency of drug-positive urine samples in individuals who had previously been shown to be resistant to standard methadone treatment. Translational studies offer a convenient approach for systematically evaluating the effects of treatment on members of a response class. For example, Heinicke, Carr, and LeBlanc (2012) used a translational approach to evaluate the effects of noncontingent reinforcement on two members of a response class and found, for the most part, decreases in both responses even though only one response was targeted for intervention (i.e., the other response was still reinforced). Future translational studies could examine the effects of other treatments on members of response-class hierarchies produced by different response–response and response– reinforcer relations. Finally, more research is needed to determine an efficient means of identifying response-class hierarchies before treatment. Treatments based on the results of a functional analysis (Iwata, Dorsey, Slifer, Bauman, & Richman, 1982/1994) may be more likely to succeed if previously established response-class hierarchies are taken into consideration and are used as the basis for extinction and differential reinforcement programs. REFERENCES Baer, D. M. (1982). The imposition of structure on behavior and the demolition of behavioral structures. In D. J. Bernstein (Ed.), Response structure and organization. The 1981 Nebraska symposium on motivation (pp. 217–254). Lincoln: University of Nebraska Press. Catania, A. C. (2007). Learning (4th interim ed.). New York, NY: Sloan. DeLeon, I. G., Fisher, W. W., Herman, K. M., & Crosland, K. C. (2000). Assessment of a response bias for aggression over functionally equivalent appropriate behavior. Journal of Applied Behavior Analysis, 33, 73– 77. doi: 10.1901/jaba.2000.33-73
81
Fisher, W., Piazza, C. C., Bowman, L. G., Hagopian, L. P., Owens, J. C., & Slevin, I. (1992). A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis, 25, 491–498. doi: 10.1901/jaba. 1992.25-491 Grow, L. L., Kelley, M. E., Roane, H. S., & Shillingsburg, M. A. (2008). Utility of extinction-induced response variability for the selection of mands. Journal of Applied Behavior Analysis, 41, 15–24. doi: 10.1901/jaba. 2008.41-15 Harding, J. W., Wacker, D. P., Berg, W. K., Barretto, A., Winborn, L., & Gardner, A. (2001). Analysis of response class hierarchies with attention-maintained problem behaviors. Journal of Applied Behavior Analysis, 34, 61–64. doi: 10.1901/jaba.2001.34-61 Heinicke, M. R., Carr, J. E., & LeBlanc, L. A. (2012). The effects of fixed-time reinforcement schedules on functional response classes: A translational study. Journal of Applied Behavior Analysis, 45, 511–526. doi: 10.1901/jaba.2012.45-511 Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1994). Toward a functional analysis of self-injury. Journal of Applied Behavior Analysis, 27, 197–209. doi: 10.1901/jaba.1994.27-197 (Reprinted from Analysis and Intervention in Developmental Disabilities, 2, 3–20, 1982) Lalli, J. S., Mace, F. C., Wohn, T., & Livezey, K. (1995). Identification and modification of a response-class hierarchy. Journal of Applied Behavior Analysis, 28, 551– 559. doi: 10.1901/jaba.1995.28-551 Mace, F. C. (1994). Basic research needed for stimulating the development of behavioral technologies. Journal of the Experimental Analysis of Behavior, 61, 529–550. doi: 10.1901/jeab.1994.61-529 Magee, S. K., & Ellis, J. (2000). Extinction effects during the assessment of multiple problem behaviors. Journal of Applied Behavior Analysis, 33, 313–316. doi: 10.1901/ jaba.2000.33-313 Mendres, A. E., & Borrero, J. C. (2010). Development and modification of a response class via positive and negative reinforcement: A translational approach. Journal of Applied Behavior Analysis, 43, 653–672. doi: 10.1901/ jaba.2010.43-653 Neef, N. A., Bicard, D. F., & Endo, S. (2001). Assessment of impulsivity and the development of self-control in students with attention deficit hyperactivity disorder. Journal of Applied Behavior Analysis, 34, 397–408. doi: 10.1901/jaba.2001.34-397 Neef, N. A., & Lutz, M. N. (2001). A brief computer-based assessment of reinforcer dimensions affecting choice. Journal of Applied Behavior Analysis, 34, 57–60. doi: 10.1901/jaba.2001.34-57 Neef, N. A., Mace, F. C., & Shade, D. (1993). Impulsivity in students with serious emotional disturbance: The interactive effects of reinforcer rate, delay, and quality. Journal of Applied Behavior Analysis, 26, 37–52. doi: 10.1901/jaba.1993.26-37
82
GRACIE A. BEAVERS et al.
Neef, N. A., Mace, F. C., Shea, M. C., & Shade, D. (1992). Effects of reinforcer rate and reinforcer quality on time allocation: Extensions of matching theory to educational settings. Journal of Applied Behavior Analysis, 25, 691–699. doi: 10.1901/jaba.1992.25-691 Neef, N. A., Marckel, J., Ferreri, S. J., Bicard, D. F., Endo, S., Aman, M. G., … Armstrong, N. (2005). Behavioral assessment of impulsivity: A comparison of children with and without attention deficit hyperactivity disorder. Journal of Applied Behavior Analysis, 38, 23–37. doi: 10.1901/jaba.2005.146-02 Neef, N. A., Shade, D., & Miller, M. S. (1994). Assessing influential dimensions of reinforcers on choice in students with serious emotional disturbance. Journal of Applied Behavior Analysis, 27, 575–583. doi: 10.1901/ jaba.1994.27-575 Shabani, D. B., Carr, J. E., & Petursdottir, A. I. (2009). A laboratory model for studying response-class
hierarchies. Journal of Applied Behavior Analysis, 42, 105–121. doi: 10.1901/jaba.2009.42-105 Shukla-Mehta, S., & Albin, R. W. (2003). From hypotheses to interventions: Applied challenges of intervening with escalating sequences of problem behavior. The Behavior Analyst Today, 4, 271–288. Retrieved from http:// www.baojournal.com/BAT%20Journal/VOL-4/BAT-4-3. pdf#page¼5 Silverman, K., Chutuape, M. A., Bigelow, G. E., & Stitzer, M. L. (1999). Voucher-based reinforcement of cocaine abstinence in treatment-resistant methadone patients: Effects of reinforcement magnitude. Psychopharmacology, 146, 128–138. doi: 10.1007/s002130051098 Received January 14, 2013 Final acceptance September 30, 2013 Action Editor, James Carr
2014, 47, 70–82
NUMBER
1 (SPRING)
PARAMETERS OF REINFORCEMENT AND RESPONSE-CLASS HIERARCHIES GRACIE A. BEAVERS GEORGIA STATE UNIVERSITY
BRIAN A. IWATA UNIVERSITY OF FLORIDA
AND
MEAGAN K. GREGORY FLORIDA INSTITUTE OF TECHNOLOGY
Shabani, Carr, and Petursdottir (2009) examined the effects of a response–response relation (effort) on the development of a response-class hierarchy using a laboratory model. Response–reinforcer relations may have similar influences. Using a similar translational approach, we examined the effects of reinforcer rate, quality, delay, and magnitude in a series of separate experiments conducted with 8 individuals with intellectual disabilities. Response-class hierarchies emerged along the dimension of rate for 3 of 6 subjects, quality for 5 of 5 subjects, delay for 2 of 8 subjects, and magnitude for 5 of 6 subjects. Key words: reinforcer dimensions, response class
in the frequency of the other responses are observed (Harding et al., 2001; Lalli, Mace, Wohn, & Livezey, 1995; Magee & Ellis, 2000; Shukla-Mehta & Albin, 2003). Results from this line of research have raised several questions about treatment of problem behavior, and, in particular, the potential for increasing more severe problem behavior when less severe problem behavior is extinguished. Noting the difficulty of conducting studies on response-class hierarchies with actual problem behavior, Shabani, Carr, and Petursdottir (2009) illustrated a laboratory model of a response-class hierarchy along the dimension of response effort. In Study 1, they first trained four children to press each of three buttons (low, medium, and high effort) separately on fixed-ratio (FR) 1 schedules of reinforcement. They then reinforced responses on all three buttons on a concurrent FR 1 schedule and observed that subjects responded primarily on the low-effort button. When responses on the low-effort button subsequently were extinguished, subjects responded mainly on
A response class consists of behaviors that are topographically different but functionally similar in that they are maintained by the same reinforcing consequences, and a hierarchical response class is one in which responses tend to occur in a particular order (Baer, 1982). The development of response-class hierarchies may be influenced by response–response relations, such as effort, or response–reinforcer relations, including reinforcer rate, quality, delay, or magnitude (Mace, 1994; Mendres & Borrero, 2010). Applied research on response-class hierarchies has proceeded from anecdotal observation, followed by a functional analysis whose results identified a common maintaining variable for all of the responses, and a final analysis in which one member of the class is extinguished while changes We thank Tara Fahmie and Amanda Rone for their assistance. Address correspondence to Brian A. Iwata, Department of Psychology, Room 114 Psychology Building, University of Florida, Gainesville, Florida 32611 (e-mail: [email protected]). doi: 10.1002/jaba.102
70
RESPONSE CLASS the medium-effort button. When responses on all buttons again were reinforced, subjects’ low-effort responding reemerged while medium- and higheffort responding decreased. In Study 2, Shabani et al. conducted a further analysis by extinguishing responses on each of the three buttons. When responses on the low-effort button were placed on extinction, subjects usually responded on the medium-effort button first. When responses on the medium-effort button were placed on extinction, subjects usually pressed the low-effort button first. When responses on both the lowand medium-effort buttons were placed on extinction, the subjects usually responded on the high-effort button. These results supported the interpretation that the three button presses had become members of the same response class and that the responses were hierarchically organized along the dimension of response effort. Mendres and Borrero (2010) replicated and extended the results of Shabani et al. with 11 college students. Mendres and Borrero operationalized effort as differences in reinforcement schedule requirements and showed that responseclass hierarchies developed under contingencies of both positive and negative reinforcement. In addition to response effort, several parametric features of reinforcement contingencies that influence operant behavior in general also may play a role in the development of response-class hierarchies. The purpose of this study was to extend previous research by examining the influences of reinforcer rate, quality, delay, and magnitude on the formation of response-class hierarchies. Although the arrangement used by Mendres and Borrero (2010) was designed to examine the effects of response effort, their manipulation of reinforcement schedules also inherently involved the dimension of reinforcer rate. We minimized differences in schedule requirements, thereby separating the effects of rate and effort. Another difference between the Shabani et al. (2009) and Mendres and Borrero studies and ours was the conditions under which responding was established initially. In their
71
studies, each response first was established under a separate arrangement. These individual histories of reinforcement may have facilitated the rapid appearance of a hierarchy later when all responses were reinforced concurrently. We wanted to see whether hierarchies would emerge naturally as a result of differential but concurrent reinforcement from the outset of training, which may reflect the way in which many response-class hierarchies actually develop.
METHOD Subjects and Setting Eight individuals, all of whom had been diagnosed with an intellectual disability, participated in one or more manipulations. Mack (40 years old), James (33 years old), Jimmy (49 years old), Rita (69 years old), Nancy (40 years old), Angela (46 years old), and Hank (49 years old) were recruited from a day-training facility for adults with intellectual disabilities; Grayson (18 years old) attended classes in a special education school. All subjects were capable of following simple instructions and performing the experimental task, and none had been reported to engage in significant problem behavior. Sessions were conducted in a quiet room at the day program or the school that contained a table, chairs, and materials needed to conduct sessions (see below). One to five sessions were conducted per day, usually 5 days per week. Response Measurement and Reliability The target response was touching one of three cards that were located on a table in front of the subject. Observers used handheld computers to collect data, which were summarized as the percentage of trials on which responses occurred on each card. An independent observer collected data during a minimum of 25% of sessions for each subject in every experimental condition. Although data were summarized as a percentage of trials, data were collected using a continuous
72
GRACIE A. BEAVERS et al.
recording procedure. Thus, a proportional reliability calculation was used in which each session was divided into continuous 10-s intervals, and observers’ records were compared on an interval-by-interval basis. The smaller number of responses was divided by the larger in each interval, and these fractions were averaged across the session. Mean interobserver agreement scores across subjects in each manipulation were as follows: reinforcer rate, 97% (range, 95% to 98%); reinforcer quality, 97% (range, 95% to 98%); reinforcer delay, 97% (range, 96% to 98%); and reinforcer magnitude, 96% (range, 92% to 98%). General Procedure A reversal design was used to evaluate the influence of each parameter of reinforcement on the development of a response-class hierarchy. In each manipulation, one parameter of reinforcement varied while the others were held constant. A session consisted of 20 trials. Each trial began with the presentation of three cards and ended when a response was made on one of the cards or when 30 s elapsed with no response to any card. The position of the cards was rotated after every trial. The intertrial interval was held constant at 30 s to equate trial duration across all conditions. Before each session, the experimenter prompted the subject to touch each card and delivered the consequence associated with that card touch in the current condition. This presession forced exposure was conducted to increase the salience of the consequences associated with each card touch and to decrease the likelihood of carryover across conditions. Preference Assessment Before each manipulation, a paired-stimulus preference assessment (Fisher et al., 1992) was conducted with nine edible items that the subject had sampled beforehand. Items were selected by the experimenter from an array of available supplies at each site and included a
variety of tastes and textures. On each trial, the experimenter presented two items on separate plates in front of the subject and asked the subject to select one of the items. The subject was allowed to consume the selected item, and the other item was removed. Trials continued until each item had been paired once with every other item, and results were summarized as the percentage of trials on which an item was selected. Items selected on 80% or more of trials were considered highly preferred and were used as reinforcers in the rate, delay, and magnitude manipulations. Additional criteria were used to select preferred stimuli in the reinforcer quality manipulation (see “Reinforcer Quality” below). Reinforcer Rate Three different colored cards were placed on the table in front of the subject; each card was paired with a different FR schedule of reinforcement (FR 1, FR 5, or FR 10). Each card also displayed the number of responses (1, 5, or 10) required to obtain reinforcement. A modified card touch was used with individuals who demonstrated difficulty in discriminating the contingencies associated with each card: Velcro tabs corresponding to the schedule requirements were affixed to each card, and subjects were required to remove the tabs to complete the schedule. If the subject touched the FR 5 or FR 10 card, the other cards were removed, and the subject had 30 s to complete the ratio before the next trial began. Contingent on ratio completion, the therapist placed one piece of the highly preferred edible item identified in the preference assessment on a plate in front of the subject (except when the response associated with that schedule was placed on extinction). The subject could consume the earned edible items at any time during the session. Baseline. During baseline, no programmed consequences were delivered for touching any of the cards or engaging in any other response. Reinforcement (all). In this condition, responses on all cards were reinforced (provided
RESPONSE CLASS the schedule requirements were met before 30 s elapsed). Reinforcement (FR 5/FR 10). In this condition, only responses on the cards associated with the FR 5 and FR 10 schedules were reinforced; responses on the FR 1 card were placed on extinction. Reinforcement (FR 10). In this condition, only responses on the FR 10 card were reinforced; responses on the FR 1 and FR 5 cards were placed on extinction. Additional training conditions. The following conditions were conducted only if a responseclass hierarchy did not emerge in the initial reinforcement (all) condition: (a) In the reinforcement (FR 5) condition, only responses on the FR 5 card were reinforced; responses on the FR 1 and FR 10 cards were placed on extinction. (b) In the reinforcement (FR 1) condition, only responses on the FR 1 card were reinforced; responses on the FR 5 and FR 10 cards were placed on extinction. Reinforcer Quality In this manipulation, three different colored cards were associated with high-, moderate-, and low-preference reinforcers, respectively. Highpreference (HP) items were those selected on 90% or more of trials in the preference assessment, moderate-preference (MP) items were those selected on between 25% and 50% of trials, and low-preference (LP) items were those either never selected or selected the least. A single-stimulus preference assessment was conducted with items from the LP group to ensure that these items had some reinforcing value. Each LP item was presented singly on five separate trials. An item that was consumed on all five trials was included as the LP item in experimental sessions. Before the session, HP and MP items, as well as MP and LP items, were paired at least 10 but no more than 20 times to ensure that the subject’s preference hierarchy was maintained in the subsequent sessions. First, HP and MP items were paired: If the subject selected the HP item
73
on the first 10 presession trials, no other HP/MP trials were conducted, and the HP item remained HP for the subsequent session. If the subject’s selection varied during the first 10 trials, 10 additional trials (20 total) were conducted, and the item selected most frequently was designated HP for the subsequent session. After the HP/MP presession trials, MP and LP items were paired using the same procedures as described for the HP/MP presession trials. After the preference hierarchy had been established, the experimental session began. During the session, contingent on a card touch, the therapist placed one piece of the corresponding edible item on a plate in front of the subject (except when the response emitted was placed on extinction). The subject could consume the earned edible items at any time during the session. Baseline. In baseline, no programmed consequences were delivered for touching any of the cards or engaging in any other response. Reinforcement (all). In this condition, touches on all cards were reinforced; that is, a touch on the HP card was followed by delivery of the HP edible item, and so on. Reinforcement (MP/LP). In this condition, only responses on the MP and LP cards were reinforced; responses on the HP card were placed on extinction. Reinforcement (LP). In this condition, only responses on the LP card were reinforced; responses on the HP and MP cards were placed on extinction. Reinforcer Delay In this manipulation, three different-colored cards were associated with three delays to reinforcement: 1 s, 10 s, and 30 s. Digital timers that displayed the corresponding delays were placed behind the 10-s and 30-s cards. Contingent on a card touch, the therapist either immediately placed one piece of the edible item on a plate in front of the subject (1-s delay) or started the timer with the delay corresponding to the card (10-s or 30-s delay) and delivered the reinforcer when the time elapsed
74
GRACIE A. BEAVERS et al.
(except when the response emitted was placed on extinction). The subject could consume the earned edible items at any time during the session. Baseline. In baseline, no programmed consequences were delivered for touching any of the cards or engaging in any other response. Reinforcement (all). In this condition, touches on all cards were reinforced; responses on the 1-s card resulted in a 1-s delay to delivery of reinforcement, and so on. Reinforcement (10 s/30 s). In this condition, only responses on the 10-s and 30-s delay cards were reinforced; responses on the 1-s delay card were placed on extinction. Reinforcement (30 s). In this condition, only responses on the 30-s delay card were reinforced, and responses on the 1-s and 10-s delay cards were placed on extinction. Additional training conditions. The following conditions were conducted only if a responseclass hierarchy did not emerge in the initial reinforcement (all) condition: (a) Only responses on the 10-s delay card were reinforced, and responses on the 1-s and 30-s delay cards were placed on extinction, or (b) only responses on the 1-s delay card were reinforced, and responses on the 10-s and 30-s delay cards were placed on extinction. Reinforcer Magnitude In this manipulation, three different-colored cards were associated with three magnitudes of reinforcement: one, two, and five pieces of an edible item. Each card also displayed the number (1, 2, 5) corresponding with the number of pieces of reinforcement. One subject (Grayson) verbally stated a preference for the number 2; thus, for him, small, medium, and large pieces of one edible item were used (the small, medium, and large pieces were one quarter, one half, and one whole edible item, respectively). Contingent on a card touch, the therapist placed the corresponding number of pieces of the edible item on a plate in front of the subject (except when the response emitted was placed on extinction). The subject
could consume the earned edible items at any time during the session. Baseline. In baseline, no programmed consequences were delivered for touching any of the cards or engaging in any other response. Reinforcement (all). In this condition, touches on all cards were reinforced with the corresponding number of pieces (or sizes) of the edible item. Reinforcement (M2/M1). In this condition, only responses on the M2 and M1 (medium and small) cards were reinforced; responses on the M5 (large) card were placed on extinction. Reinforcement (M1). In this condition, only responses on the M1 (small) card were reinforced, and responses on the M2 and M5 (medium and large) cards were placed on extinction. Additional training conditions. The following conditions were conducted only if a responseclass hierarchy did not emerge in the initial reinforcement (all) condition: (a) Only responses on the M2 (medium) card were reinforced, and responses on the M1 and M5 (small and large) cards were placed on extinction, or (b) only responses on the M5 (large) card were reinforced, and responses on the M1 and M2 (small and medium) cards were placed on extinction.
RESULTS Results for each subject were considered indicative of the development of a response-class hierarchy if (a) in the reinforcement (all) condition, the majority of responses occurred on the card associated with the most favorable parameter of reinforcement (i.e., the densest rate, the highest quality, the shortest delay, or the largest magnitude), and (b) when responding on one or more of the cards was placed on extinction, the majority of responses occurred on the card associated with the next most favorable parameter of reinforcement. In other words, relative response probabilities observed over the course of experimental conditions were used as the basis
RESPONSE CLASS for determining that a hierarchy of responses had developed (Catania, 2007). Figure 1 shows representative results for individuals for whom a response-class hierarchy emerged as predicted by the parameters of reinforcement in each manipulation: Jimmy in the rate manipulation, James in the quality manipulation, Mack in the delay manipulation, and Grayson in the magnitude manipulation. In baseline, all subjects exhibited zero or near-zero responding on all cards, with the exception of Grayson, who responded on all cards before ceasing altogether. When reinforcement was delivered for responses on all cards in the reinforcement (all) condition, all subjects allocated the majority of their responding to the card associated with the most favorable parameter of reinforcement. Mack never responded on the other cards, whereas James showed some initial responding on the other cards that decreased across sessions, and Jimmy and Grayson occasionally responded on the other cards throughout the condition. When responding on the card associated with the most favorable parameter of reinforcement was extinguished, and only responses on the other cards were reinforced, all subjects shifted their responding to the card associated with the next most favorable parameter of reinforcement. When responses on all cards were again reinforced, all subjects again responded primarily on the card associated with the most favorable parameter of reinforcement. When reinforcement was only delivered following responses on the card associated with the least favorable parameter of reinforcement and responses on the other cards were extinguished, all subjects responded mainly on the card associated with reinforcement. In the final reinforcement (all) condition, all subjects again allocated the majority of their responses to the card associated with the most favorable parameter of reinforcement. Other subjects for whom a response-class hierarchy emerged as predicted by the parameters of reinforcement in each manipulation showed
75
results closely resembling those in Figure 1 with one exception: During the reinforcement (LP) condition of the quality manipulation, three subjects’ responding varied from the pattern exhibited by James (Figure 1). Grayson initially responded on the card associated with the LP item, but his responding decreased to zero on all cards as the condition continued. Nancy and Rita never responded on the card associated with the LP item at any point during the reinforcement (LP) condition. Additional evidence of hierarchical development was obtained from analyses of subjects’ responding on the first trial of the first session of each condition after baseline. That is, the larger the percentage of trials on which subjects immediately switched to the card as predicted by a given parameter of reinforcement, the more likely a hierarchy had formed. The percentage of trials on which an immediate switch to the predicted card was observed for all subjects for whom a response-class hierarchy emerged in the rate, quality, delay, and magnitude manipulations was 66.7%, 80%, 40%, and 64%, respectively. Thus, the strongest additional evidence of hierarchical development was associated with the quality manipulation. Figure 2 shows typical results for individuals for whom a response-class hierarchy did not emerge as predicted by the parameters of reinforcement: Angela in the rate manipulation, Hank in the delay manipulation, and Rita in the magnitude manipulation. In baseline, two different patterns were observed: Angela and Hank did not respond on any of the cards, whereas Rita responded evenly on all cards despite the absence of all programmed consequences. In the initial reinforcement (all) condition, all three subjects allocated responding equally to all cards. Because these subjects did not show the emergence of a response-class hierarchy during the initial reinforcement (all) condition, additional training conditions were conducted to ensure that subjects discriminated the contingencies associated with each card. When each parameter of
GRACIE A. BEAVERS et al.
76 Baseline
Sr+(All)
Sr+(FR 5/FR 10)
Sr+(All)
Sr+(FR 10)
Sr+(All)
100 FR 5
FR 1
80 60
FR 10 JIMMY (RATE)
40 20 0
Baseline
Sr+(All)
Sr+(MP/LP)
Sr+(All)
Sr+(LP)
Sr+(All)
100 80
HP
60
CARD TOUCH (% TRIALS)
40
JAMES (QUALITY)
LP MP
20 0
100
Baseline
Sr+(All)
Sr+(10 s/30 s)
1s
80
Sr+(All)
Sr+(30 s)
Sr+(All)
10 s
60 MACK (DELAY)
30 s
40 20 0
100
Baseline
Sr+(All)
80
Sr+(Med/Small)
Sr+(All)
Sr+(Small)
Sr+(All)
Large Med
60 GRAYSON (MAGNITUDE)
40 Small 20 0 5
10
15
20
25
30
35
40
SESSIONS Figure 1. Representative data for individuals who showed emergence of response-class hierarchies as predicted by the rate, quality, delay, and magnitude manipulations. Srþ ¼ reinforcement.
reinforcement was isolated in a separate condition (three values for rate, delay, and magnitude), subjects responded according to the prevailing contingency. That is, when responding on only one card at a time was reinforced (responding on the other cards was extinguished), all subjects
showed clear response differentiation. However, when reinforcement (all) was reimplemented, these subjects again responded indifferently. Other subjects for whom a response-class hierarchy did not emerge as predicted by the parameters of reinforcement in each manipulation
RESPONSE CLASS 100
Baseline
Sr+(All)
Sr+(FR 10)
Sr+(FR 5)
77 Sr+(FR 1)
Sr+(All)
80 60
FR 5
40 ANGELA (RATE)
20 FR 10 FR 1
CARD TOUCH (% TRIALS)
0 Baseline
Sr+(All)
Sr+(1 s)
Sr+(10 s)
Sr+(30 s)
Sr+(All)
100 80 10 s
1s
60 40 20
HANK (DELAY)
30 s 0
100
Baseline
Sr+(All)
Sr+(M 1)
Sr+(M 2)
Sr+(M 5)
Sr+(All)
80 60
M2
40 RITA (MAGNITUDE)
20 M5 M1 0 5
10
15
20
25
30
35
40
SESSIONS Figure 2. Representative data for individuals who failed to develop response-class hierarchies as predicted by the rate, delay, and magnitude manipulations. Srþ ¼ reinforcement.
showed results closely resembling those in Figure 2 with two exceptions, both seen in Figure 3. Nancy showed emergence of a hierarchy but not of the type predicted. In the initial reinforcement (all) condition of the rate manipulation, Nancy responded more frequently on the less favorable FR 5 card than on the FR 1 card. When responses on each card (FR 1, FR 5, FR 10) were isolated in separate reinforcement conditions, Nancy’s responding showed orderly effects. When the reinforcement (all) condition was reinstated,
however, she again allocated more responses to the FR 5 card. When responding only on the FR 1 and FR 10 cards was reinforced, Nancy allocated the majority of her responding to the FR 1 card. In the final reinforcement (all) condition, Nancy again responded primarily on the FR 5 card. Grayson initially showed the typical emergence of a response-class hierarchy in the delay manipulation. In the initial reinforcement (all) condition, he responded primarily on the card associated with the 1-s delay. When responses
GRACIE A. BEAVERS et al.
78
100
Baseline
Sr+ (All)
Sr+ (FR 1)
Sr+ Sr+ (FR 5) (FR 10)
Sr+ (All)
Sr+ (FR 1/FR 10)
Sr+ (All) NANCY (RATE)
80 FR 5
CARD TOUCH (% TRIALS)
60
FR 1
40
FR 10
20 0 Sr+ (All)
BL
Sr+ Sr+ Sr+ Sr+ (10 s/30 s) (All) (1 s/30 s) (1 s)
Sr+ (All)
100
GRAYSON (DELAY)
80 60 40
1s 10 s 30 s
20 0 5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
SESSIONS Figure 3. Data showing emergence of atypical response-class hierarchies. Srþ ¼ reinforcement.
only on the 10-s and 30-s delay cards were reinforced, Grayson responded mainly on the 10s delay card. However, when we reinstated the reinforcement (all) condition, Grayson’s responding on the 1-s delay card did not increase as it had in the prior reinforcement (all) condition; instead, he continued to respond primarily (and later exclusively) on the 10-s delay card. The next condition revealed an even more unusual pattern of results. When responses only on the 1-s and 30-s delay cards were reinforced, Grayson responded mainly on the 30-s delay card. We then reinforced responses only on the 1-s delay card, and differential responding was observed. In the final reinforcement (all) condition, Grayson again responded primarily on the 10-s delay card. Table 1 summarizes the results for all subjects in all manipulations, the order in which subjects were exposed to each manipulation, and the total number of sessions conducted within each manipulation. Six subjects participated in the reinforcer rate manipulation (M assessment duration ¼ 45 sessions). James, Jimmy, and Mack
showed development of a response-class hierarchy, whereas Angela, Nancy, and Rita did not. Five subjects (Grayson, James, Mack, Nancy, and Rita) participated in the quality manipulation (M assessment duration ¼ 43 sessions), and all five subjects’ responding was indicative of the development of a response-class hierarchy. All subjects participated in the delay manipulation (M assessment duration ¼ 42 sessions). James and Mack showed development of a response-class hierarchy; Angela, Grayson, Hank, Jimmy, Nancy, and Rita did not. Of the six subjects who participated in the magnitude manipulation (M assessment duration ¼ 42 sessions), Grayson, James, Jimmy, Mack, and Nancy developed a response-class hierarchy, but Rita did not. DISCUSSION In a series of separate manipulations, we determined whether response-class hierarchies emerged naturally given disparities in the rate, quality, delay, and magnitude of reinforcement
RESPONSE CLASS Table 1 Summary of Results Subject
Rate
Quality
Delay
Magnitude
Angela Grayson Hank James Jimmy Mack Nancy Rita Total
N (1) 35 — — Y (1) 38 Y (1) 35 Y (1) 33 N (1) 92 N (1) 35 3/6
— Y (1) 47 — Y (3) 38 — Y (2) 36 Y (2) 37 Y (2) 59 5/5
N (2) 31 N (2) 52 N (1) 37 Y (2) 47 N (3) 48 Y (3) 35 N (4) 48 N (4) 37 2/8
— Y (3) 38 — Y (4) 34 Y (2) 61 Y (4) 30 Y (3) 55 N (3) 32 5/6
Note. Y indicates development of a response-class hierarchy, N indicates either no or atypical development of a response-class hierarchy, and a dash indicates the specified manipulation was not conducted with that subject. Numbers in parentheses indicate the order in which the subject was exposed to a manipulation, and the numbers after the parentheses indicate the number of sessions conducted in each manipulation.
for different responses. Response-class hierarchies emerged along the dimension of rate for three of six subjects, quality for five of five subjects, delay for two of eight subjects, and magnitude for five of six subjects. Ideally, each parameter would have been evaluated with each subject; unfortunately, setting constraints prevented this. Nevertheless, these results extend the research of Shabani et al. (2009) and Mendres and Borrero (2010) by demonstrating that response-class hierarchies were sensitive to each of the parameters of reinforcement evaluated for at least some subjects and that, for these subjects, hierarchical responding emerged as a function of parametric differences in reinforcement from the outset of training. The most consistent results were seen with the quality and magnitude manipulations. The robust effects of reinforcer quality on choice allocation have been demonstrated previously by Neef and colleagues in a series of studies that showed that reinforcer quality may override the influence of both reinforcer rate and delay for some individuals (Neef, Bicard, & Endo, 2001; Neef & Lutz, 2001; Neef et al., 2005; Neef,
79
Mace, & Shade, 1993; Neef, Mace, Shea, & Shade, 1992; Neef, Shade, & Miller, 1994). Another possible explanation was the salience of the response–reinforcer relations associated with each card in the manipulations: Although we attempted to enhance the salience of the contingencies in the rate and delay manipulations (i.e., different numbers of Velcro tabs for some subjects in the rate manipulation; timers facing all subjects in the delay manipulation), it is possible that the visual presence of different edible items in the quality manipulation and different numbers of edible items in the magnitude manipulation were stronger discriminative stimuli for these contingencies. Unlike Mendres and Borrero (2010), we did not observe consistent development of responseclass hierarchies in our rate manipulation. This discrepancy might be due to our use of a smaller range of schedules (FR 1, 5, and 10) than those used by Mendres and Borrero (FR 5, 15, and 25). We selected smaller ratios to minimize differences in reinforcer delay across ratios; however, it should be noted that even at smaller ratios, an increase in reinforcer delay is inherent to a larger schedule. Other differences across studies, such as the populations from which subjects were drawn, or the fact that we initially exposed subjects to all schedules concurrently, also may have been responsible for differences in results. Future research could determine whether these variables have any systematic influence on responding. Although hierarchies emerged in 15 of 25 cases in our study, they did not emerge (at least, based on the most favorable parameters of reinforcement) in 10 cases, despite histories of exposure to separate training conditions. That is, if a response-class hierarchy did not emerge in the initial concurrent reinforcement condition, additional training procedures had no effect on the future development of a response-class hierarchy. In spite of relatively clear differences in conditions of reinforcement, some subjects responded indifferently when all responses were
80
GRACIE A. BEAVERS et al.
reinforced or seemed to develop unusual preferences (e.g., Nancy preferred the FR 5 schedule over the FR 1 schedule) based on unknown factors. A similar situation might exist when treatment for problem behavior involves delivery of reinforcement for alternative behavior under an arrangement that is more favorable than that for problem behavior, but problem behavior persists nevertheless (e.g., DeLeon, Fisher, Herman, & Crosland, 2000). Thus, although more favorable conditions of reinforcement often may be sufficient to alter response allocation, other factors may determine the conditions under which a response-class hierarchy emerges and is maintained. One subject (Grayson) admitted a preference for a specific number, which, if unknown, may have affected the development of a response-class hierarchy in the magnitude manipulation. Unfortunately, it was not always possible to identify through client verbal report what extraneous variable may have influenced some subjects’ atypical responding. If the influential variable can be identified, then manipulations can be made to restore optimal responding. Results of studies such as the present one have several implications for practice, especially when attempting to extinguish problem behavior, because more severe problem behavior may emerge. For example, Lalli et al. (1995), after conducting a functional analysis of a child’s screaming, aggression, and self-injury and finding that all responses were maintained by negative reinforcement (escape from demands), systematically placed each response on extinction. When screaming was extinguished, the latency to aggression and self-injury decreased. However, when screaming was reinforced, no instances of self-injury and very few instances of aggression occurred. Thus, if a client is referred for treatment of a particular topography of problem behavior, but caregivers report a history of occasional occurrences of other topographies of problem behavior, care should be taken to ensure appropriate treatment strategies are in place in
the event that other, potentially more severe, topographies of problem behavior emerge over the course of treatment. Response–response and response–reinforcer relations also may affect the conditions under which replacement behavior is established or maintained. For example, Grow, Kelley, Roane, and Shillingsburg (2008) investigated the emergence of appropriate responses during extinction of problem behavior. Following a baseline condition in which inappropriate behavior was reinforced and no programmed consequences were delivered for other responses, the authors extinguished inappropriate behavior and delivered the consequences that maintained inappropriate behavior contingent on another response previously emitted by the subject. Results showed that these untrained alternative responses occurred at higher rates when problem behavior was placed on extinction, despite the lack of explicit training of the alternative response. The authors suggested that both the inappropriate and appropriate responses were members of the same response-class hierarchy, and that the inappropriate response occurred most frequently in the natural environment because of some response- or reinforcer-related variable. One limitation of laboratory analyses of response classes is that results may not accurately reflect what happens in the real world. In particular, our use of a trial-based arrangement was obviously different from what often occurs in real-world settings. Future research might attempt to replicate these results using free-operant procedures similar to those of Shabani et al. (2009) and Mendres and Borrero (2010). Because we isolated variables singly to identify the influence of each, we cannot speak to the effects of combinations of variables, which may covary simultaneously in the natural environment. Future research may examine the influence of combinations of variables on the development of response-class hierarchies, because this information may be useful in selecting optimal treatments for problem behavior. Results of
RESPONSE CLASS studies on response-class hierarchies also may have applications for other clinical populations. For example, Silverman, Chutuape, Bigelow, and Stitzer (1999) showed that larger magnitudes of voucher reinforcement were more effective than smaller magnitudes in reducing the frequency of drug-positive urine samples in individuals who had previously been shown to be resistant to standard methadone treatment. Translational studies offer a convenient approach for systematically evaluating the effects of treatment on members of a response class. For example, Heinicke, Carr, and LeBlanc (2012) used a translational approach to evaluate the effects of noncontingent reinforcement on two members of a response class and found, for the most part, decreases in both responses even though only one response was targeted for intervention (i.e., the other response was still reinforced). Future translational studies could examine the effects of other treatments on members of response-class hierarchies produced by different response–response and response– reinforcer relations. Finally, more research is needed to determine an efficient means of identifying response-class hierarchies before treatment. Treatments based on the results of a functional analysis (Iwata, Dorsey, Slifer, Bauman, & Richman, 1982/1994) may be more likely to succeed if previously established response-class hierarchies are taken into consideration and are used as the basis for extinction and differential reinforcement programs. REFERENCES Baer, D. M. (1982). The imposition of structure on behavior and the demolition of behavioral structures. In D. J. Bernstein (Ed.), Response structure and organization. The 1981 Nebraska symposium on motivation (pp. 217–254). Lincoln: University of Nebraska Press. Catania, A. C. (2007). Learning (4th interim ed.). New York, NY: Sloan. DeLeon, I. G., Fisher, W. W., Herman, K. M., & Crosland, K. C. (2000). Assessment of a response bias for aggression over functionally equivalent appropriate behavior. Journal of Applied Behavior Analysis, 33, 73– 77. doi: 10.1901/jaba.2000.33-73
81
Fisher, W., Piazza, C. C., Bowman, L. G., Hagopian, L. P., Owens, J. C., & Slevin, I. (1992). A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis, 25, 491–498. doi: 10.1901/jaba. 1992.25-491 Grow, L. L., Kelley, M. E., Roane, H. S., & Shillingsburg, M. A. (2008). Utility of extinction-induced response variability for the selection of mands. Journal of Applied Behavior Analysis, 41, 15–24. doi: 10.1901/jaba. 2008.41-15 Harding, J. W., Wacker, D. P., Berg, W. K., Barretto, A., Winborn, L., & Gardner, A. (2001). Analysis of response class hierarchies with attention-maintained problem behaviors. Journal of Applied Behavior Analysis, 34, 61–64. doi: 10.1901/jaba.2001.34-61 Heinicke, M. R., Carr, J. E., & LeBlanc, L. A. (2012). The effects of fixed-time reinforcement schedules on functional response classes: A translational study. Journal of Applied Behavior Analysis, 45, 511–526. doi: 10.1901/jaba.2012.45-511 Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1994). Toward a functional analysis of self-injury. Journal of Applied Behavior Analysis, 27, 197–209. doi: 10.1901/jaba.1994.27-197 (Reprinted from Analysis and Intervention in Developmental Disabilities, 2, 3–20, 1982) Lalli, J. S., Mace, F. C., Wohn, T., & Livezey, K. (1995). Identification and modification of a response-class hierarchy. Journal of Applied Behavior Analysis, 28, 551– 559. doi: 10.1901/jaba.1995.28-551 Mace, F. C. (1994). Basic research needed for stimulating the development of behavioral technologies. Journal of the Experimental Analysis of Behavior, 61, 529–550. doi: 10.1901/jeab.1994.61-529 Magee, S. K., & Ellis, J. (2000). Extinction effects during the assessment of multiple problem behaviors. Journal of Applied Behavior Analysis, 33, 313–316. doi: 10.1901/ jaba.2000.33-313 Mendres, A. E., & Borrero, J. C. (2010). Development and modification of a response class via positive and negative reinforcement: A translational approach. Journal of Applied Behavior Analysis, 43, 653–672. doi: 10.1901/ jaba.2010.43-653 Neef, N. A., Bicard, D. F., & Endo, S. (2001). Assessment of impulsivity and the development of self-control in students with attention deficit hyperactivity disorder. Journal of Applied Behavior Analysis, 34, 397–408. doi: 10.1901/jaba.2001.34-397 Neef, N. A., & Lutz, M. N. (2001). A brief computer-based assessment of reinforcer dimensions affecting choice. Journal of Applied Behavior Analysis, 34, 57–60. doi: 10.1901/jaba.2001.34-57 Neef, N. A., Mace, F. C., & Shade, D. (1993). Impulsivity in students with serious emotional disturbance: The interactive effects of reinforcer rate, delay, and quality. Journal of Applied Behavior Analysis, 26, 37–52. doi: 10.1901/jaba.1993.26-37
82
GRACIE A. BEAVERS et al.
Neef, N. A., Mace, F. C., Shea, M. C., & Shade, D. (1992). Effects of reinforcer rate and reinforcer quality on time allocation: Extensions of matching theory to educational settings. Journal of Applied Behavior Analysis, 25, 691–699. doi: 10.1901/jaba.1992.25-691 Neef, N. A., Marckel, J., Ferreri, S. J., Bicard, D. F., Endo, S., Aman, M. G., … Armstrong, N. (2005). Behavioral assessment of impulsivity: A comparison of children with and without attention deficit hyperactivity disorder. Journal of Applied Behavior Analysis, 38, 23–37. doi: 10.1901/jaba.2005.146-02 Neef, N. A., Shade, D., & Miller, M. S. (1994). Assessing influential dimensions of reinforcers on choice in students with serious emotional disturbance. Journal of Applied Behavior Analysis, 27, 575–583. doi: 10.1901/ jaba.1994.27-575 Shabani, D. B., Carr, J. E., & Petursdottir, A. I. (2009). A laboratory model for studying response-class
hierarchies. Journal of Applied Behavior Analysis, 42, 105–121. doi: 10.1901/jaba.2009.42-105 Shukla-Mehta, S., & Albin, R. W. (2003). From hypotheses to interventions: Applied challenges of intervening with escalating sequences of problem behavior. The Behavior Analyst Today, 4, 271–288. Retrieved from http:// www.baojournal.com/BAT%20Journal/VOL-4/BAT-4-3. pdf#page¼5 Silverman, K., Chutuape, M. A., Bigelow, G. E., & Stitzer, M. L. (1999). Voucher-based reinforcement of cocaine abstinence in treatment-resistant methadone patients: Effects of reinforcement magnitude. Psychopharmacology, 146, 128–138. doi: 10.1007/s002130051098 Received January 14, 2013 Final acceptance September 30, 2013 Action Editor, James Carr
