The Perception-of-Relationships Test (PORT) and Bricklin Perceptual Scales (BPS), 1961-2002
Current and New Empirical Data on 3,880 Cases
Barry Bricklin, Ph.D.
Gail Elliot, Ph.D.
Evidence-to-Conclusions Models: the Goal of Science
Knowing reliability and validity numbers, along with information about previously examined groups, cannot themselves tell a decision-maker whether a test is addressing the precise conceptual world in which there is interest. What is required is the entire chain of reasoning that links evidence to conclusions. This paper, which describes parts of an ongoing 7-year project, offers an evidence-to-conclusions chain for the Bricklin Perceptual Scales (BPS) and the Perception-of-Relationships Test (PORT) (Bricklin, 1984; 1989).
The PORT and BPS form the research-derived, data-based part of a comprehensive system: A Comprehensive Custody Evaluation Standard System, ACCESS (Bricklin & Elliot, 1995). The tests measure the comfort and efficiency with which a child and his or her caretakers exchange a wide variety of interpersonal and emotional information within multiple family contexts. They are not meant to assign parents to legal categories. Such assignments are made by the court and depend upon information not obtainable from scientific models.
The following model will be used to organize the evidence-to-conclusions chain for the PORT and BPS. It consists of four tiers. The first tier consists of concepts, the second of principles. Principles state the relations among concepts. Empirical Equivalents, abbreviated EEs, define what one looks for in the world of sensory experience that exemplifies a concept. What one chooses as the EEs to represent any concept cannot be determined by a scientific process. All that can be claimed is that other choices may have created better or worse (i.e., different) results. As Einstein put it, the ways we draw boundaries around our sensory worlds, the physical as well as the mental, are “free creations of the human mind” (Einstein, 1936; 1954; 1984). Validation refers to the degree to which the relations among the EEs of the concepts correspond to the relations among the concepts as stated in the principles.
The predictor concepts of the tests aim to yield information on how effectively and comfortably a caretaker and child can exchange all manners of information (concerning the caretaker’s offering and modeling of competency skills, emotional support, consistency, etc.) given the child’s age, developmental status and idiosyncratic needs. At a test level, the concept’s EEs are represented by the test items, a respondent’s responses to the items and the test’s scoring system.
Perhaps the most important tier for a decision-maker involves the EEs chosen to prove what a test claims to measure. Here we refer to the precise way the independent validating-criteria are selected and measured. It is with these data that the decision-maker can question whether the test is measuring the decision-area of concern.
In the new data to be presented, each mental health professional who made a validity criterion designation had to have continuing contact either with the family of a tested child, or with one who had such contact. Since we wanted ecologic validity, each validity designator was instructed to use all of the test, documentary, observation and other clinical/life-history information available (except for PORT or BPS scores). This included numerous consultations with the mental health professionals who had continuing contact with each child and his or her family over the time-spans involved. The exact questions asked of those who made criterion designations were as follows. “Which parent (or other caretaker) overall offers and models competency skills, emotional support, discipline and structure in ways that are best suited to a child’s ability to use these responses effectively and comfortably?” This was followed by a question asking whether any difference between the parents was narrow or substantial. (“Narrow” was interpreted to mean the parents were approximately equal.) We further asked whether both parents did not meet even minimal standards of “good-enough” parenting.
This yields the following choices: A is substantially better than B; B is substantially better than A; A and B are approximately equal; neither A nor B is even minimally adequate. At no point did we, in this study, use the term “parent of choice” as was done in some of our earlier research, since the term was confused with that for a parent serving in some legally defined role. Out tests measure the comfort and effectiveness with which a parent and child can exchange all manners of information. Other criteria enter the picture when a decision-maker assigns a parent to a legal category.
2-1. Sex: 797 females; 784 males
2-2. Age: Mean age 7.76; SD=0.17
2-3. SES: Low-Middle to High-Middle
2-4. Race: 98 percent Caucasian; 2 percent all other
3. BPS Normative Data (1964-1997), n=2,389
3-1. Sex: 1202 females; 1,187 males
3-2. Age: Mean age 8.94; SD=2.40
3-3. SES: Low-Middle to High-Middle
3-4. Race: 98 percent Caucasian; 2 percent all others
4. PORT Test-Retest Stability Data (1962-1997), n=21 (More data are given later.)
4-1. If the Task-Difference-Score (TDS) (over a 6-month interval) was 0 or 1, there was a 10 percent chance the PORT Parent-of-Choice (POC) would shift. If the TDS was 2 or more, the chance of POC shift was 2 percent.
5. BPS Test-Retest Stability Data (1964-1997), n=20; n=33
5-1. If the Item-Difference-Score (IDS) (over a 1-week interval, n=20 and a 6-month interval, n=33) was 0, 1 or 2, there was a 10 percent chance of POC shift. If the IDS was 3 or more, there was a 2 percent chance of shift.
Please note again that the validity designations, presented next, were matched to the degree of refinement generally required in the legal system: parent A>B; parent B>A; Parent A @ Parent B; Neither A nor B is a good choice. PORT and BPS data are summarized to yield a designation matching the first 3 of these requirements. The system of which they are a part—ACCESS—addresses the last choice. Those who made independent designations never did so on the bases of PORT or BPS data.
Please note, then, that the criterion designation categories yielded by the PORT and BPS do not yield a normal or symmetrical distribution. Hence, percent-of-agreement is a reasonable way to express our results, and are inherently more user-friendly to many of those who would make decisions on their bases. Further, it is a lot easier for non-statisticians to comprehend percent-of-agreement rates than product-moment correlations, where fairly large numbers frequently signify modest predictive accuracy. A test-suggested Parent-of-Choice, abbreviated POC, is never, by itself or as part of our comprehensive system, used either to address legal ultimate issues or to assign a caretaker to a legal category.
6. PORT Validity Data (1961-1997), n=1,381
6-1. The percent-of-agreement rate is listed following the sample size. Structured task problem-solving by children with access to both parents, observed from behind a one-way screen by two psychologists (1961), n=30, 90 percent; courtroom judges (1964-1981), based on all data available, n=45, 89 percent; agreement with BPS choices (1964-1981), n=23, 83 percent; courtroom judges (1981-1985), based on all data available, n=42, 95 percent; agreement with BPS choices (1981-1983), n=30, 84 percent; two psychologists, based on family therapy notes plus consultation with relevant therapists with families seen over two- to five-year intervals (1980-1985), n=30, 93 percent; courtroom judges (1986-1990), based on all data available, n=76, 93 percent; independent psychologists based on all clinical (except for PORT and BPS scores) and life-history data available (1995-1997), n=1,038, 89 percent.
7. BPS Validity Data (1964-1997), n=2,279
7-1. Agreement with PORT choices (1964-1981), n=23, 83 percent; two psychologists, based on family therapy notes plus consultation with relevant therapists with families seen over two- to seven-year intervals (1980-1983), n=21, 100 percent; courtroom judges (1980-1983), n=30, 90 percent; “Would” questionnaire choices (a “disguised” semi-projective test, asking what Mommy/Daddy would do in certain situations e.g., “You get a bad mark on a test”) (1980-1983), n=23, 87 percent; PORT choices (1981-1983), n=30, 84 percent; courtroom judges based on all available information (1984-1990), n=179, 96 percent; independent psychologists based on all clinical and life-history data available (1988), n=141, 97 percent; independent psychologists based on all clinical and life-history data available (1992-1995), n=1,765, 88 percent; independent psychologists based on all clinical and life-history data available (1995-1997), n=67, 87 percent.
8. PORT Normative Data (1997-2002), n=127
8-1. Sex: 61 females; 66 males
8-2. Age: Mean age 7.87; SD=2.101
8-3. SES: Low-Middle to Upper-Middle
8-4. Race: 92 percent Caucasian; 8 percent all other
9. BPS Normative Data (1997-2002), n=93
9-1. Sex: 47 females; 46 males
9-2. Age: Mean age 7.88; SD=1.473
9-3. SES: Low-Middle to Upper-Middle
9-4. Race: 92 percent Caucasian; 8 percent all other
10. PORT Test-Retest Stability (1997-2002), n=127
10-1 If the TDS is 0 or 1, there is a 21 percent chance of POC shift over an 8-month interval. If the TDS is 2 or more, there is a 3 percent chance of POC shift. PORT results were highly stable over the 8-month interval. Only seven percent of the cases showed a POC-shift, and 66 percent of these changes occurred where the Task-Difference Scores were zero or one. If the TDS is 2 or more, the likelihood that the POC will not change is 97 percent.
11. BPS Test-Retest Stability (1997-2002), n=93
11-1. If the IDS is 0, 1, 2 or 3, there is a 19 percent chance of POC shift over an 8-month interval. If the IDS is 4 or more, there is a 3 percent chance of POC shift. If expressed as a percentage, the BPS test-retest reliability would be about 93 percent.
12. PORT Validity Data (1997-2002), n=127
12-1. Independent psychologists based on all clinical and life history data available (n=127):
Future Validity (8-month interval) = 89 percent (pretest POC compared to posttest criterion designation)
Concurrent Validity = 91 percent (posttest POC compared to posttest criterion designation)
13. BPS Validity Data (1997-2002), n=93
13-1. Independent psychologists based on all clinical and life history data available (n=93):
Future Validity (8-month interval) = 87 percent
Concurrent Validity = 91 percent
17. We investigated whether a POC is likely to change more readily as a child grows older. It has been our observation that as children grow older (11 years and up), they become far more self-centerdly interested in “shopping around for the best deal” in terms of which household to live in. Their own circles of friends are increasing and solidifying, and they know which house will offer them the greater range of the freedoms they seek. This is often the household in which the rules are more relaxed, where they can stay up later, watch more television, go out more frequently, etc. This hypothesis was not confirmed.
18. PORT Task I POC reflects the caretaker in relation to whom the child is acquiring the most overall assets.
18-1. Agreement rate (n=72) 76 percent
19. PORT Task II POC reflects the caretaker with whom the child is most comfortable in a situation where he or she is alone with that person.
19-1. Agreement rate (n=72) 63 percent
20. PORT Task V POC reflects which caretaker the child believes contributes the most to continuing family unity and cooperation.
20-1. Agreement rate (n=72) 91 percent
21. What Clinical Hypotheses are Suggested by Empirical Data From 3,880 Custody Cases?
21-1. The generally high rates of agreement between test-based selections for POC and those arrived at by a variety of independent sources suggest one can reasonably measure the ongoing nature of the comfort and efficiency with which a child and caretaker exchange information. The validation data were collected not only from multiple and independent sources, but were the product of data gathered over substantially long intervals, ranging from four months to seven years.
21-2. Given how closely our concurrent validity numbers matched those obtained from a future-validity paradigm, we can hypothesize that the more concurrent validity methods are ecologic in nature, the more it is reasonable to assume they will yield data comparable to future validation data.
21-3. These particular data remained stable over at least an 8-month period (Tables 1 and 2).
21-4. One can detect score patterns that suggest when family (primarily dyadic) subsystems are likely to be unstable over time. One of these is when a child is not able to recognize any useful parenting differences between his or her parents (Table 3).
21-5. The presence of an ADHD pattern in a child does not seem to create instability over time in how the child assigns value to his or her caretakers (Table 4).
21-6. When one parent is better than the other in multiple-perspective-taking-skills and a child has continually complained about his or her post-divorce situation, we have a red flag that the dyadic family subsystems of which that child is part will be unstable (Table 5).
21-7. At least in so far as our sample was concerned, a child’s increasing age does not seem to red flag instability. This was counter to what we believed would be true (Table 6).
21-8. PORT Task I seems to reasonably reflect the parent who is the greater source of overall assets to the child.
21-9. Although the evidence is weaker here, a child’s response to PORT Task II seems to indicate the parent with whom the child is most comfortable in a situation where the child is alone with that parent.
21-10. PORT Task V is strongly predictive of the parent whom the child perceives as contributing the most toward family unity.
21-11. Nonverbal data both for adults as well as children were more accurate in predicting the validating criteria than were interview and other consciously-sourced data. Although some of these data were not generated in the current study, it is worth repeating that children’s consciously sourced opinions on POC-choices had only about a fifty percent agreement rate with those of experts. This was true even with older children, 14 through 16 years of age. Children’s nonverbal responses had agreement rates with the experts in the ninety- to one-hundred percent range. In our data, the interview-based opinions of high conflict custody parents are relatively useless. One parent-questionnaire had 86 questions about which parent the child turns to in times of need. In adversarial parent cases, the typical scores were 86 to zero. These scores had no relationship whatsoever to data generated by experts. On the other hand, nonadversarial parents (from intact families) achieved agreement rates with experts that were quite reasonable (70 to 80 percent range).
21-12. All of our data collectively suggest that the meaning of a communication is best understood by the response it engenders. The point is that the latter is often unrelated to what the communicator thinks is the sent-message. For example, one of our tests categorizes some of a child’s responses into “perception of caretaker competency” and “perception of caretaker emotional supportiveness.” It was quite interesting to note the hugely divergent subjective experiences that children perceive as “emotional supportiveness.” For example, disorganized children who have difficulty with planning prioritized sequences often perceive as the more “emotionally supportive” caretaker the one who is most organized. In a group referenced (normative) paradigm, this caretaker may or may not be seen as “emotionally warm.” Such children obviously feel more emotionally supported by a parent who reduces chaos in their lives rather than by one who exudes what others might call “warmth.”
References and Bibliography
Bricklin, B. (1984). The Bricklin perceptual scales: Child-perception-of-parents series. Furlong, PA: Village Publishing Inc.
Bricklin, B. (1989). Perception-of-relationships-test (PORT). Furlong, PA: Village Publishing Inc.
Bricklin, B. (1992). Data-based tests in custody evaluations. American Journal of Family Therapy, 20, 254-265.
Bricklin, B. (1995a). Getting optimum performance from a mental health professional. Family Law Quarterly, 29, 7-17.
Bricklin, B. (1995b). The custody evaluation handbook: Research based solutions and applications. New York: Brunner-Mazel.
Bricklin, B. (1998). The sequence of steps and critical targets of a comprehensive custody evaluation. In G. Koocher, J. Norcross, & S. Hill (Eds.), The psychologist's desk reference. New York: Oxford University Press.
Bricklin, B. (1999). The contributions of psychological tests to custody-relevant evaluations. In R. Galatzer-Levy & L. Kraus (Eds.), The scientific basis of custody decisions. Colorado Springs, CO: Wiley Press.
Bricklin, B. (2001). Nostalgia, integration, validity issues and generative power. Journal of Personality Assessment, 76(2), 194-199.
Bricklin, B., & Bricklin, P. M. (1999). Custody data as decision-theory information: Evaluating a psychological contribution by its value to a decision-maker. Clinical Psychology: Science and Practice., 6(3), 339-343.
Bricklin, B., & Elliot, G. (1995). ACCESS: A comprehensive custody evaluation standard system. Furlong, PA: Village Publishing, Inc.
Bricklin, B., & Elliot, G. (2001). The Bricklin family of custody instruments: Updated validity, reliability and normative data. Furlong, PA: Village Publishing, Inc. and www.villagepublishinghomepage.com
Bricklin, B., & Elliot, G. (in press). Parental time-share plans. In L. Gunsberg & P. Hymowitz (Eds.), Divorce and custody: Contemporary developmental psychoanalytic persepctives. Washington, DC: American Psychological Association Books.
Einstein, A. (1936). Physics and reality. The Journal of the Franklin Institute, 221(3), 1-34.
Einstein, A. (1954). Ideas and opinions. New York, NY: Crown.
Einstein, A. (1984). Out of my later years. New York, NY: Kensington.
Ekman, P. (1985). Telling lies. New York, NY: Norton.
Ekman, P., & O’Sullivan, M. (1991). Who can catch a liar? American Psychologist, 46(9), 913-920.
Horney, K. (1945). Our inner conflicts. New York, NY: Norton
Krauss, D.A., & Sales, D.B. (2000). Legal standards, expertise and experts in the resolution of contested child cases. Psychology, Public Policy, and Law, 6(4), 843-879.
Miller, R. B. (1998). Epistemology and psychotherapy data: The unspeakable, unbearable, horrible truth. Clinical Psychology: Science and Practice, 5(2), 242-250.
Piotrowski, Z. A. (1957). Perceptanalysis. New York: Maxmillan.
Sales, B.D. (2000). Legal standards, expertise and experts in the resolution of contested child custody cases. Psychology, Public Policy, and Law, 6(4), 843-879.