Abstract
We present a new way to understand how people perceive situations involving other people, situations that could be considered part of the everyday. The approach is Mind Genomics, which assesses the response of people to short, systematically varied vignettes about situations and other people. The responses to these vignettes are deconstructed into the part-worth contribution of the component elements that the vignette comprises, showing the ‘algebra of the mind.’ The deconstruction also is done on response time to the vignettes, showing the ability of the elements to engage attention when the respondent makes a judgment. When Mind Genomics is applied to descriptions of family life under stress, the data suggest that some elements are linked to predicted violence, others are not. Women appear to be more sensitive than men to the individual elements. Three different mind-sets emerged with different perceived ‘triggers’ to predicted family violence, with each mind-set encompassing both men and women: Mind-Set 1 – no specific warning; Mind-Set 2 – Sensitive to the economy; Mind-Set 3 – Family has problems. We present the PVI (personal viewpoint identifier) as a technique to assign new people to these mind-sets.
Introduction
Violence against the other sex, especially in marriage, is not new. Stories of murder and abuse fill the newspapers, the magazines, and the Internet news of today (late 2019.) Before today’s overwhelming plethora of news, violence by males against females, especially spouses and other family members, occupied a great deal of attention, from those in the news, but of course even more telling, from writers and poets. One cannot read the famous poem, My Last Duchess, by the 19th Century British poet, Robert Browning without a shudder when one realizes how easy it was to kill one’s spouse. And of course, the popular 1965 Rock n Roll song by Herman’s Hermits, hints at England’s royal lady-killer, King Henry VIII, transformed to a 1960’s idiom of a man with a broken heart.
What is popular in literature only reflects what is the common situation in everyday life. The literature in sociology and psychology is replete with studies about violence and anger. Violence against one’s spouse is dealt with in many publications, with the aspects dissected, studied, statistically analyzed and reports issued. Violence seems to be endemic to the relations, starting even in courtship [1]. The spousal violence continues, even into the 60’s [2]. Violence emerges when the woman ends up supporting the man [3]. Of course, alcoholism plays a role [4], but so does religion [5]. Violence comes from many quarters, but many studies have focused on gender and marriage [6, 7, 8].
The foregoing represents just a bit of the available material on violence in the home. These studies focus on both surveys and discussions with individuals. What is lacking is a sense of the richness of the family life through discussion, an absence promoted by the rigidity of the scientific method, but the absence filled by clinicians and social workers. The key issue is to make this topic come alive by merging the rigor of science with the immediacy of storytelling.
Violence in the home is especially relevant because it is common and riveting to those involved. Although there seems to be very little academically oriented literature recounting the actual ‘story’ of the abuse, the Internet provides a repository of such personal studies in a number of websites, such as:
- https://www.getdomesticviolencehelp.com/domestic-violence-stories
- www.hiddenhurt.co.uk/domestic_violence_stories.html
- https://www.domesticshelters.org/articles/true-survivor-stories
It may be that websites are more conducive to people ‘telling their story’ in their own language. In contrast, the scientific community has made its information almost unobtainable, except to those schooled in the scholastic tradition, and able to cut through the jargon and statistics to understand what exactly is happening
Exploratory studies through Mind Genomics
This study explores the mind of ‘people’ by having them evaluate different vignettes about violence, vignettes that have been systematically varied, with the components of the vignette, the element, having a richness that is missing from surveys
A review of the scientific literature suggests that many of the studies involving human judgment are done in a manner which is slow, expensive, requiring teams of researchers, and extensive, rigorous statistical analysis. The statistical analysis is often of the type known as ‘inferential,’ with the objective to confirm or to falsify an ingoing hypothesis, with the hypothesis developed from theory.
Mind Genomics presents to the world of science a different approach, not grounded in theory and confirming or falsifying hypotheses [9]. Rather, Mind Genomics can be liked to an exploration of decisions, using cognitively meaningful stimuli, and dealing with issues of the every day. Mind Genomics can be likened to a new cartographical exercise of a land. Mind Genomics works by presenting vignettes to the respondents, with these vignettes comprising combinations of elements or messages to which a respondent can relate. The respondent reads the vignette and responds to the combination. The research approach is analogous to the MRI, which takes multiple pictures of tissue from different vantage points, and then combines these into a picture of the tissue.
The research in this study embodies the Mind Genomics paradigm, dealing with the very important issue of family violence. The objective is to understand a third-party’s estimate of either violence or peace at home occurring when a specific situation is presented, and then to assess the likelihood that each specific element is correlated either with violence or with a peaceful home, respectively, two opposite sides of the scale.
Mind Genomics combines the person with emotion and meaningful description of behavior, i.e., cognitively rich test stimuli. Mind Genomics obtains ratings from the response of people to vignettes about a situation, similar that presented in literature, storytelling, or song. The vignette paints a picture of a situation. The respondent is then asked to judge some aspect of the situation, such as projected violence or projected happiness, based upon what is read. Through this approach it now becomes possible to understand the mind of the person, either the one who is undergoing the experience, or the one who is hearing/reading about the experience. Both points of view differ dramatically from the almost lifeless array of statistics describing a situation. Mind Genomics combines the vividness of experience with numbers, probing the inner mind of the person exposed to the situation, first-hand or second-hand.
The Mind Genomics approach
The Mind Genomics approach is designed to be exploratory, affordable, iterative, and scalable. This set of objectives in the design means that there are certain simple aspects of the study:
- Exploratory. As suggested above, Mind Genomics does not work by confirming or disconfirming a hypothesis extant in the scientific literature. Rather, the exploration means taking new ideas from every-day experience and exploring them to find out the degree to which people respond positively or negatively to them.
- Affordable. Mind Genomics is set up to be a so-called DIY, Do it yourself system. The researcher needs access to an APP on the proper machine (Android or Kindle), the ideas (for the researcher), and a convenient source of respondents.
- Iterative. Mind Genomics is set up to return the data in easy-to-read formats (PowerPoint® for presentation, Excel® for data analysis. The data return in a matter of a few hours. A new study can be launched a few hours later, after the results from the first study are digested. Furthermore, the results are easy to understand, and set up to promote further exploration with the same tool. With the iterative approach the researcher can do as many as 4–6 studies in a 24-hour period, each study building upon the previous study.
- Scalable. Almost anyone can use Mind Genomics to explore problems. The system is scalable across people, but also across different aspects of a topic, by the same researcher. Within a matter of a week or two, the enterprising researcher can conduct 10–20 studies, exploring the different facets of a topic.
Raw materials
The origin of this study was the focus by author Peer on the causes of violence against women, the fact that so much is known, yet so little. When random people were asked by author Moskowitz about the topic ‘What do you think causes spousal violence,’ very few people could provide an answer quickly. There was no sense of a well-recognized phenomenon, violence, connected with the daily life of people, other than general statistical compilations, available in the literature.
The benefit of a Mind Genomics study is the degree to which it takes any topic and reduces that topic to a set of common aspects, experienced in the everyday. Thus, the elements shown in Table 1 represent the way a person might conceive of the nature of spousal violence. A Mind Genomics is not meant to be exhaustive, but rather introductory, approachable, and in some ways focuses on a very specific topic. When this notion of ‘cartography’ is recognized and accepted, the position of Mind Genomics advances to a useful, early-stage way of understanding a topic from the mind of people.
Table 1. The raw materials for the study, comprising four questions about the conditions of a family, and the four answers to each question.
|
Question A: What is the current situation of the person |
A1 |
The local economy is stressed and in recession |
A2 |
The local economy is growing |
A3 |
The children are having problems |
A4 |
The couple are having long term problems |
|
Question B: What is the local situation |
B1 |
Companies are firing employees |
B2 |
Companies are hiring but people working long hours |
B3 |
It’s in middle of winter … Christmas |
B4 |
It’s summer time |
|
Question C: What does the woman do |
C1 |
The lady starts searching for a job to help out |
C2 |
The lady is having problems with finances |
C3 |
The husband is having job troubles |
C4 |
The husband is sad and depressed |
|
Question D: What happens afterward |
D1 |
The family time is shorter together |
D2 |
The family all eat at different times |
D3 |
The wife wants to talk but the husband does not |
D4 |
The husband wants to talk but the wife does not |
The reader will see the approach in Table 1, showing the four questions (which tell a story), and the four answers to each question. As we read the answers or elements, we should keep in mind that the answers are concrete and simple. When exploring a topic, we can learn a great deal from four simple questions which tell a story, and from the pattern of responses to the 16 answers. The results in this study should reveal a variety of new-to-the-world patterns about domestic violence, based simply on the different ways that people respond to these unambiguous stimuli.
With the inputs shown in Table 1, Mind Genomics creates combinations of answers, so-called vignettes. An example of a vignette appears in Figure 1.
Figure 1. Example of a vignette as presented to the respondent.
Each respondent evaluated 24 vignettes. The vignettes were constructed according to an experimental design, with the property that a vignette comprised at most one answer from each question, but often had no answers from either one or two of the questions. Thus, the vignettes comprised either two, three, or four answers, the so-called elements. Furthermore, each respondent evaluated a unique set of combinations. The underlying structure of the combinations was maintained, but the specific combinations differed from one respondent to another.
To the respondent, the combinations might seem to be random, but the reality is the exact opposite. The experimental design prescribes the combinations. The objective is to present combinations of elements or answers (without the questions), obtain ratings from the respondents who evaluate these combinations, and then deconstruct the ratings into the separate contribution from each element. In this way the respondent is unable to ‘game’ the system by providing politically correct answers. It is virtually impossible to detect the underlying pattern. As a result, the respondent simply relaxes, and gives responses which are more intuitive, and fundamentally less ‘edited.’ In the words of experimental psychologist Daniel Kahneman, the Mind Genomics approach calls into play ‘System 1’ thinking, the fast, almost automatic thinking that we use daily in our lives, when we don’t have to make rational calculations [10].
A sense of the underlying experimental design can be gotten from looking at the schematic in Table 2, which presents the structure of the first eight vignettes for Respondent #1. The respondent does not, of course, see the underlying structure, but rather the actual combinations, presented on the computer as in Figure 1, or restructured to fit on the screen of a smartphone.
Table 2. Structure of the first eight vignettes for Respondent #1, the conversion to binary for statistical analysis, and the deconstruction of the ratings and response time.
Vignette |
Vig1 |
Vig2 |
Vig3 |
Vig4 |
Vig5 |
Vig6 |
Vig7 |
Vig8 |
Design |
|
|
|
|
|
|
|
|
A |
4 |
4 |
2 |
2 |
0 |
1 |
1 |
0 |
B |
4 |
3 |
2 |
1 |
1 |
3 |
4 |
4 |
C |
2 |
2 |
4 |
1 |
3 |
0 |
1 |
4 |
D |
1 |
2 |
2 |
2 |
4 |
1 |
2 |
1 |
Binary |
|
|
|
|
|
|
|
|
A1 |
0 |
0 |
0 |
0 |
0 |
1 |
1 |
0 |
A2 |
0 |
0 |
1 |
1 |
0 |
0 |
0 |
0 |
A3 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
A4 |
1 |
1 |
0 |
0 |
0 |
0 |
0 |
0 |
B1 |
0 |
0 |
0 |
1 |
1 |
0 |
0 |
0 |
B2 |
0 |
0 |
1 |
0 |
0 |
0 |
0 |
0 |
B3 |
0 |
1 |
0 |
0 |
0 |
1 |
0 |
0 |
B4 |
1 |
0 |
0 |
0 |
0 |
0 |
1 |
1 |
C1 |
0 |
0 |
0 |
1 |
0 |
0 |
1 |
0 |
C2 |
1 |
1 |
0 |
0 |
0 |
0 |
0 |
0 |
C3 |
0 |
0 |
0 |
0 |
1 |
0 |
0 |
0 |
C4 |
0 |
0 |
1 |
0 |
0 |
0 |
0 |
1 |
D1 |
1 |
0 |
0 |
0 |
0 |
1 |
0 |
1 |
D2 |
0 |
1 |
1 |
1 |
0 |
0 |
1 |
0 |
D3 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
D4 |
0 |
0 |
0 |
0 |
1 |
0 |
0 |
0 |
Rating |
|
|
|
|
|
|
|
|
9-Point Rating |
1 |
5 |
7 |
9 |
7 |
5 |
3 |
7 |
Binary – Violence |
1 |
0 |
101 |
100 |
100 |
0 |
0 |
100 |
Binary – Happy |
100 |
0 |
0 |
0 |
0 |
0 |
100 |
0 |
Response time |
9.0 |
3.3 |
3.3 |
2.3 |
2.8 |
3.0 |
2.4 |
2.3 |
Executing the study
Each respondent receives the invitation to participate, and is instructed to read the vignette, and to rate it on the 9-point scale.
Here is a set of snapshots of families. Please read the full snapshot and tell us what will happen within the foreseeable future. Read the whole snapshot. Is it going to be peaceful or do you sense some family violence brewing?
What will happen in the foreseeable future with this family?
1=peace and love … 9=some violence
The respondent then read each of 24 unique vignettes. The respondent rated vignette on the above 9-point scale. The respondent was then instructed to fill out an open-ended question about violence (results not presented here.) The entire process took approximately 4–5 minutes.
Basic data transformation
The experimental design itself must be transformed to a binary no/yes, as shown in Table 2. Only with a binary scale (absent/present) is it feasible to understand the part-worth contribution of every element. In turn, the 9-point scale can be used as a dependent variable, but experience has shown that most people, researchers included, have a difficult time understanding what the scale points mean. Sometimes this difficulty in understand is addressed by labelling each of the nine scale points, a task which itself is fraught with difficulties. An easier way, taken from the world of consumer research, converts the nine-point scale to a binary scale, 0 or 100. Managers find it easy to understand the binary scale and know what to do with a ‘no’ or a ‘yes’ answer.
The conventional way to divide the scale creates three regions for the scale; 1–3, 4–6, and 7–9, respectively. Then the following conventions is invoked:
Ratings of 7–9 are assumed to represent ‘violence,’ and ratings 1–6 are assumed to reflect the lack of violence. For this new variable, ‘violence’, we convert ratings of 1–6 to 0, and ratings of 7–9 to 100. We then add a small random number (<10–5). The small random number ensures that that the regression analysis will ‘run’ on the binary-transformed data, even when the respondent confines all of the ratings either to the lower portion of the scale (1–6, transformed to 0), or confines all of the ratings to the upper portion of the scale (7–9 transformed to 100, 1–6 transformed to 0). The small random number provides just enough variability in the dependent to ensure that the OLS (ordinary least=squares) regression ‘does not crash,’
Analysis – What drives violence versus happiness – total panel?
The basic analysis in Mind Genomics is OLS (ordinary least-squares) regression, made possible by the ingoing structure of the vignettes for each individual respondent. Every respondent evaluated 24 carefully constructed vignettes, ensuring that at the individual level all 16 elements or answers to the questions, are statistically independent of each other. Most of the vignettes are different from each other, so that the combination of all the vignettes covers a great deal of the ‘design space.’
We combine all the data from the 50 respondents, creating a database of 1200 vignettes (50 × 24 = 1200). We run two OLS regressions. The first relates the presence/absence of all 16 variables to the binary value of ‘violence’, corresponding to the ratings 7–9 on the original 9-point scale, but now becoming the value 100 on the binary scale for violence. The second OLS regression relates the presence/absence of all 16 variables to the violence of ‘happiness’ corresponding to the ratings of 1–3 on the original 9-point scale.
Table 3 shows the coefficients for the two equations. The equation is expressed as (Binary Rating) = k0 + k1(A1) + k2(A2) + … k16(D4).
Table 3. Parameters of the model for the Total Panel relating the presence / absence of the 16 elements to predicted violence (Ratings of 7–9 converted to 100), and to predicted happiness (Ratings of 1–3 converted to 100).
|
|
Violence |
Happiness |
|
Additive constant |
27 |
12 |
C4 |
The husband is sad and depressed |
6 |
-3 |
B1 |
Companies are firing employees |
5 |
4 |
A1 |
The local economy is stressed and in recession |
4 |
-2 |
D3 |
The wife wants to talk but the husband does not |
3 |
-4 |
B3 |
It’s in middle of winter .. Christmas |
3 |
9 |
A4 |
The couple are having long term problems |
1 |
0 |
C3 |
The husband is having job troubles |
1 |
-3 |
A3 |
The children are having problems |
1 |
1 |
D1 |
The family time is shorter together |
-1 |
-2 |
D2 |
The family all eat at different times |
-1 |
-2 |
D4 |
The husband wants to talk but the wife does not |
-1 |
-2 |
B2 |
Companies are hiring but people working long hours |
-1 |
5 |
C2 |
The lady is having problems with finances |
-2 |
2 |
B4 |
It’s summer time |
-4 |
10 |
A2 |
The local economy is growing |
-5 |
6 |
C1 |
The lady starts searching for a job to help out |
-11 |
4 |
The additive constant, k0, is the estimated value of the binary response in the absence of elements. All vignettes comprised a minimum of two and a maximum of four elements. Consequently, the additive constant is an estimated parameter. Nonetheless, the additive constant has value in because it gives a sense of baseline interest or baseline feeling, in the absence of elements.
As noted above, the experimental designs ensure that all 16 elements or answers are statistically independent of each other, allowing the absolute coefficients to be estimated. That is, the values of the coefficients are all relative to 0. A coefficient of 10 is twice as high as a coefficient of 5. Furthermore, the transformation of the scale to binary strengthens the mathematic property. The coefficient of 10 means that in the absence of elements, 10% of the responses will be suggest ‘violence’ (7–9). The coefficient of 5 means that in the absence of elements, 5% of the responses, half the number as before, will suggest ‘violence.’ The absolute value of the coefficient means that the coefficients can be compared from study to study, with different topics and different respondents. The ratio scale properties generated by the binary transformation means that one can relate ratio changes in the coefficients (or properly coefficient + additive constant) to external behaviors. The negative coefficient means that when the element is added to the vignette, the percent of response suggesting ‘violence’ will be removed. Thus, when the coefficient is -10, then adding the element to a vignette will decrease the percent suggesting ‘violence’ by 10%. The coefficients are additive and subtractive.
From thousands of such experiments, a set of rules of thumb have emerged about the value of the coefficients, based upon observations of the data, and knowledge about what happens in the external world. The table below provides these guidelines, which are qualitative in nature. There are no fixed values, but rather a shading of importance, so that the higher the positive number the more important the element.
- Coefficient of 15 or higher Extremely important, major signal
- Coefficient 8–15 Important to very important
- Coefficient of 0–8 From irrelevant to almost important
- Coefficient 0 to -6 From irrelevant to almost important
- Coefficient from -6 to lower Important
We interpret the parameters of the model for violence (ratings of 7–9 converted to 100).
- The Additive constant is 27, meaning that there is a low likelihood of predicting violence in the absence of elements. We can compare this to say the purchase intent for pizza on the same type of 9-point scale, albeit with different anchors (definitely not buy … definitely buy). The additive constant for pizza is around 60.
- The elements for predicted violence are low. There is only one which even approaches potential meaningfulness, C4 (The husband is sad and depressed).
- We move now to the parameters of the model for happiness (ratings of 1–3 converted to 100.)
- The additive constant is 12, meaning that there is very little in the way of predicted happiness in the absence of elements.
- Two elements emerge as strong drivers of predicted happiness, both related to season:
- B4 (It’s summer time)
- B3 (It’s the middle of winter … Christmas)
Genders react differently when predicting violence, but similarly when predicting happiness
Respondents profiled themselves in term of gender. When we divide the data sets by gender and estimate the two models by gender (predicted violence versus predicted happiness), we find dramatic differences in the models for predicted violence, but similar models for predicted happiness (Table 4).
Table 4. Parameters of the model for males versus females relating the presence / absence of the 16 elements to predicted violence (Ratings of 7–9 converted to 100), and to predicted happiness (Ratings of 1–3 converted to 100).
|
|
Female |
Male |
Female |
Male |
|
|
Violence |
Happiness |
||
|
Additive constant |
16 |
39 |
11 |
13 |
C4 |
The husband is sad and depressed |
15 |
-4 |
-3 |
-2 |
B1 |
Companies are firing employees |
13 |
-3 |
2 |
6 |
B3 |
It’s in middle of winter … Christmas |
10 |
-5 |
8 |
11 |
A1 |
The local economy is stressed and in recession |
4 |
4 |
-4 |
0 |
D3 |
The wife wants to talk but the husband does not |
6 |
0 |
-3 |
-5 |
A3 |
The children are having problems |
2 |
0 |
0 |
2 |
A4 |
The couple are having long term problems |
3 |
-1 |
2 |
-2 |
C3 |
The husband is having job troubles |
3 |
-2 |
0 |
-6 |
D4 |
The husband wants to talk but the wife does not |
1 |
-2 |
-1 |
-2 |
D2 |
The family all eat at different times |
0 |
-2 |
-4 |
-1 |
A2 |
The local economy is growing |
-8 |
-3 |
9 |
2 |
D1 |
The family time is shorter together |
4 |
-5 |
-3 |
-1 |
C2 |
The lady is having problems with finances |
1 |
-6 |
2 |
2 |
B2 |
Companies are hiring but people working long hours |
4 |
-7 |
2 |
9 |
B4 |
It’s summer time |
1 |
-8 |
9 |
11 |
C1 |
The lady starts searching for a job to help out |
-10 |
-12 |
3 |
4 |
Table 5. Parameters of the model for the three age groups relating the presence / absence of the 16 elements to predicted violence (Ratings of 7–9 converted to 100), and to predicted happiness (Ratings of 1–3 converted to 100).
|
|
Age50+ |
Age30–49 |
A19–29 |
Age50+ |
Age30–49 |
A19–29 |
|
Violence |
Happiness |
|
||||
|
Additive constant |
22 |
27 |
31 |
2 |
3 |
37 |
C4 |
The husband is sad and depressed |
13 |
7 |
-5 |
0 |
-9 |
-6 |
B1 |
Companies are firing employees |
11 |
5 |
-1 |
3 |
8 |
-4 |
A1 |
The local economy is stressed and in recession |
1 |
8 |
3 |
-3 |
8 |
-12 |
D3 |
The wife wants to talk but the husband does not |
4 |
2 |
8 |
2 |
-10 |
-9 |
B2 |
Companies are hiring but people working long hours |
-3 |
-1 |
5 |
3 |
12 |
1 |
B3 |
It’s in middle of winter …Christmas |
1 |
6 |
4 |
9 |
13 |
8 |
D1 |
The family time is shorter together |
3 |
-7 |
3 |
4 |
-6 |
-6 |
D2 |
The family all eat at different times |
2 |
0 |
-2 |
2 |
-6 |
-6 |
B4 |
It’s summer time |
-5 |
-1 |
-2 |
8 |
19 |
3 |
A4 |
The couple are having long term problems |
4 |
3 |
-6 |
-1 |
2 |
-1 |
A2 |
The local economy is growing |
-7 |
-1 |
-6 |
8 |
6 |
3 |
A3 |
The children are having problems |
1 |
6 |
-6 |
0 |
5 |
-2 |
D4 |
The husband wants to talk but the wife does not |
2 |
3 |
-8 |
2 |
-3 |
-5 |
C3 |
The husband is having job troubles |
6 |
6 |
-15 |
-2 |
-6 |
2 |
C2 |
The lady is having problems with finances |
7 |
1 |
-18 |
3 |
2 |
1 |
C1 |
The lady starts searching for a job to help out |
-7 |
-8 |
-23 |
7 |
-1 |
6 |
Predicted violence
- Additive constant – lower for females, higher for males (16 vs 39). The difference suggests that the prediction of violence by female respondent occurs for specific situations. In contrast, for males the additive constant is much higher, suggesting that they predict violence without needing to have specifics.
- Women predict that the violence will occur in different situations, the most surprising of which is the expectation of violence during Christmas time.
The husband is sad and depressed
Companies are firing employees
It’s in middle of winter … Christmas
Predicted happiness
- Additive constant is very low, 11 for females, 13 for 13
- Surprisingly, women are divided on winter and Christmas, with females reacting to
It’s in the middle of winter … Christmas
The local economy is growing
It’s summer time
- Males are happy as well, with both season and a growing economy
It’s in the middle of winter … Christmas
Companies are hiring but people are working long hours
It’s summer time
Predicted Violence
- The additive constants, prediction of violence without other information, are low, with the additive constant lowest for age 50+ (value = 22), and the additive constant modestly higher for age 19 to 29 (value =31)
- There are age differences in what drives predicted violence.
- The oldest respondents, age 50+ predict that violence will occur with the husband sad and depressed, and the companies firing employees.
- The middle group age predict that violence will occur when the local economy is stressed and in recession
- The young respondents don’t predict violence will occur in these bad economic times but predict violence will occur when the wife wants to talk but the husband does not.
- We conclude from this pattern that the older respondents, age 50+, see violence as externally driven, whereas the young respondents, age 19–29 see violence as interaction driven.
Predicted happiness
- The additive constants, base expectations without elements, vary dramatically across ages. The older respondents (age 50+ and age 30 to 49) see no basic happiness. It’s all a matter of the specifics. The younger respondents, age 19 to 29, in contrast, feel that happiness is all around.
- The oldest respondents feel that happiness is a function of the time, whether Christmas or the summer.
- The middle group, age 30 to 39, show some answers which make sense (e.g., companies are honoring, summer time, winter time), but also some answers which don’t make sense (companies are firing employees’ the local economy is stressed and in recession). It could be that this age group feels that the hard times will bring the couple together, rather than eventuate in violence.
- The youngest group age 19 to 39 feel that happiness will emerge with the Christmas season, but not with the summer season.
Response time and engagement with the elements in the vignette
For more than a century, researchers have searched for ‘objective’ correlates of psychological processes. The notion that the information provided by people was not acceptable to many researchers, who believed, whether correctly or not, that only ‘objective’ physical measures could tell the truth about what a person perceives or thinks. The history of these approaches traces back to the original research on reaction time in the Leipzig laboratory of Wilhelm Wundt [11], and moves on to physiological measures of human reactions, whether GSR (galvanic skin response, electrical conductance of the skin), electromyography (muscle currents), then EEG (electroencephalographs and brain waves), culminating in such methods as fMRI [12, 13]. There are other more recently introduced methods, such as the implicit association test [14].
Response time, the earliest measure and perhaps the most frequently used measure, may shed additional light on the nature of the way people respond to the elements or answers embedded in the vignettes. Mind Genomics has the distinct benefit that the test stimuli, the elements, are themselves cognitively meaningful. It’s not a case of having to infer ‘what about the stimulus’ makes the respondent process it more quickly or more slowly. One can simply look at the response times to the different elements, using deconstruction method below, and ask whether there is something common about those elements taking longer to process, versus those elements processed more quickly.
The Mind Genomics computer program measured the response time to the different vignettes. It then eliminated all vignettes requiring more than 9 seconds to rate, under the assumption that in these Mind Genomics studies, rarely does a respondent stop to consider a vignette for longer than a few seconds. The Mind Genomics program also eliminates all vignettes tested in the first position, with the rationale that at the start of the experiment respondents don’t know what to do.
Figure 2 shows the distribution of response times, with the abscissa spaced logarithmically. The important thing is the relatively large number of vignettes requiring more than four seconds to process. In many comparable studies, albeit with mundane topics like food, we do not see such long response times. There may be a difference in the way people read serious vignettes, such as the vignettes here, versus ‘fun vignettes’ of other topics.
Figure 2. Distribution of response times for the study on predicted family violence. The distribution has been trimmed to eliminate the responses from the vignette evaluated in the first position, and vignettes registering 9 seconds or longer to evaluate.
The analysis of response times follows the standard approach, involving OLS (ordinary least-squares) regression. The equation is written without the additive constant, based upon the ingoing assumption that in the absence of a vignette with elements, there is no response. All vignettes, however, except those tested first, are included in the OLS regression, with all vignettes of response times 9 or more seconds truncated to 9.
The equation is expressed as: Response Time = k1(A1) + k2(A2) … k16(D4)
The analysis was performed in the precisely the same way as the regression analyses for the ratings. That is, the relevant group was identified, and all the appropriate vignettes from everyone in the relevant group was put into a single data file, accessed by the OLS regression package.
The coefficients represent the number of tenths of seconds that can be ascribed to each element. The OLS regression deconstructs the response time, estimating the number of tenths of seconds for each element. In the analyses we will look at those response times for individual elements of 1.5 seconds or more. The cut-off of 1.5 seconds is arbitrary, allowing us to get a sense of those elements which strongly engaged the respondents. It is important to keep in mind that these socially relevant topics appear to be generating longer response times than the more typical business and marketing topics run in the same fashion, with the same type of respondents. It may be that respondents pay more attention to socially relevant topics.
The response times for the 16 elements as shown in Table 6 suggest a continuum with response times of 1.0–1.5 seconds. Keep in mind that all response times over 9 seconds or longer were eliminated as suggesting that the respondent might be doing other things. The data do not suggest a pattern. The most engaging elements, those with the longest response times, talk about the couple, about the economy, and about the woman having problems.
Table 6. Response times for the 16 elements, estimated from the data of the Total Panel.
|
Response time for the total panel |
Total |
A4 |
The couple are having long term problems |
1.5 |
B2 |
Companies are hiring but people working long hours |
1.5 |
C2 |
The lady is having problems with finances |
1.5 |
D4 |
The husband wants to talk but the wife does not |
1.5 |
A1 |
The local economy is stressed and in recession |
1.3 |
B3 |
It’s in middle of winter … Christmas |
1.3 |
D3 |
The wife wants to talk but the husband does not |
1.3 |
A2 |
The local economy is growing |
1.2 |
B1 |
Companies are firing employees |
1.2 |
C1 |
The lady starts searching for a job to help out |
1.2 |
D2 |
The family all eat at different times |
1.2 |
C4 |
The husband is sad and depressed |
1.1 |
D1 |
The family time is shorter together |
1.1 |
A3 |
The children are having problems |
1.0 |
B4 |
It’s summer time |
1.0 |
C3 |
The husband is having job troubles |
1.0 |
By gender
When we divide the respondents by gender, we see radical differences. The most important result is that men do not find the elements engaging, at least when we operationally define the term ‘engaging’ as a response time of 1.5 seconds (Table 7.)
Table 7. Response times for the 16 elements, estimated from the data broken out by gender.
|
Response time in seconds – by gender |
Male |
Female |
D4 |
The husband wants to talk but the wife does not |
1.0 |
2.0 |
A4 |
The couple are having long term problems |
1.3 |
1.7 |
B2 |
Companies are hiring but people working long hours |
1.3 |
1.7 |
C2 |
The lady is having problems with finances |
1.4 |
1.6 |
A1 |
The local economy is stressed and in recession |
1.0 |
1.6 |
D3 |
The wife wants to talk but the husband does not |
0.9 |
1.6 |
B3 |
It’s in middle of winter … Christmas |
1.1 |
1.5 |
B1 |
Companies are firing employees |
0.9 |
1.5 |
D2 |
The family all eat at different times |
1.1 |
1.4 |
C1 |
The lady starts searching for a job to help out |
1.0 |
1.4 |
D1 |
The family time is shorter together |
0.8 |
1.4 |
A2 |
The local economy is growing |
1.2 |
1.2 |
C4 |
The husband is sad and depressed |
1.2 |
1.1 |
B4 |
It’s summer time |
0.9 |
1.1 |
C3 |
The husband is having job troubles |
0.8 |
1.1 |
A3 |
The children are having problems |
1.1 |
1.0 |
Table 7. Response times for the 16 elements, estimated from the data broken out by age group.
|
Response time in seconds – by age |
Age 50+ |
Age 30–49 |
Age 19–29 |
B2 |
Companies are hiring but people working long hours |
2.0 |
1.4 |
0.8 |
D4 |
The husband wants to talk but the wife does not |
1.9 |
1.4 |
1.2 |
A4 |
The couple are having long term problems |
1.9 |
1.4 |
0.7 |
D3 |
The wife wants to talk but the husband does not |
1.9 |
1.2 |
0.7 |
C2 |
The lady is having problems with finances |
1.8 |
2.1 |
0.7 |
B3 |
It’s in middle of winter … Christmas |
1.6 |
1.2 |
0.9 |
C1 |
The lady starts searching for a job to help out |
1.6 |
1.2 |
0.7 |
B1 |
Companies are firing employees |
1.6 |
1.1 |
0.8 |
D1 |
The family time is shorter together |
1.5 |
1.3 |
0.6 |
A1 |
The local economy is stressed and in recession |
1.5 |
1.0 |
1.2 |
D2 |
The family all eat at different times |
1.4 |
1.5 |
0.9 |
C4 |
The husband is sad and depressed |
1.2 |
1.4 |
0.8 |
A3 |
The children are having problems |
1.2 |
1.1 |
0.5 |
A2 |
The local economy is growing |
1.1 |
1.5 |
1.0 |
C3 |
The husband is having job troubles |
0.9 |
1.3 |
0.7 |
B4 |
It’s summer time |
1.1 |
0.5 |
1.3 |
Males
The most engaging element is
The lady is having problems with finances.
The least engaging elements are
The wife wants to talk but the husband does not
Companies are firing employees
It’s summer time
The family time is shorter together
The husband is having job troubles
Females
There are many engaging elements. The fact that 8 of the 16 elements are engaging to women suggest that women are simply more attentive than men to the topic of violence versus happiness.
The husband wants to talk but the wife does not
The couple are having long term problems
Companies are hiring but people working long hours
The lady is having problems with finances
The local economy is stressed and in recession
The wife wants to talk but the husband does not
It’s in middle of winter … Christmas
Companies are firing employees
Age group
Respondents age 59+
The oldest respondents focus primarily about the issues between the members of the couple, but also react to the economy (companies are hiring but people working long hours). That element might be a signal for problems that emerge between the husband and wife.
Companies are hiring but people working long hours
The husband wants to talk but the wife does not
The couple are having long term problems
The wife wants to talk but the husband does not
The lady is having problems with finances
It’s in middle of winter … Christmas
Companies are firing employees
The lady starts searching for a job to help out
Respondents age 30–49
The most engaging element is the practical issue of finances. The elements are more practical.
The lady is having problems with finances
The family all eat at different times
The local economy is growing
Respondents age -29
None of the elements engaged them. They appear to be disinterested in the topic, or at least don’t pay much attention.
Mind Sets
One of the key tenets of Mind Genomics is that in any topic area involving judgment and decision-making, there are different groups, mind-sets, showing divergent patterns of what is important. The ideal situation, but one quite rare, is that these mind-sets are congruent with some easy-to-define and measure characteristic or set of characteristics of the respondent. Most of psychological and sociological research discovering groups with different points of view, e.g., voting for political parties, attempt to understand these differences within the framework of the standard ways to divide people. Thus, it is not unusual to see voting patterns broken out by age, gender, market, income, education, work, and so forth. Indeed, the world of analytics attempts to predict these mind-set-driven behaviors from some predictive model using easy to measure variables.
In the world of Mind Genomics, the discovery of these basic groups is straightforward, requiring simply one or several studies of the type performed here, and statistical methods to cluster together individuals with similar patterns of coefficients [15]. Individuals with similar patterns are assumed to belong to the same ‘mind genome’ for the topic. The creation of the mind genome is a simple statistical analysis, once the relevant experiment has been run. In this respect Mind Genomics holds the advantage of generating easy to interpret ‘mind genomes’ from simple experiments. The reason for the simplicity is that the experiment deals with the topic itself, and the test stimuli are all relevant. One need not array an analytic armory to discover the ‘mind genomes,’ which emerge readily from these focused experiments.
The procedure for uncovering mind genomes follows these eight steps.
- Array the vector of all 16 elements for a given respondent as a one line in a data base.
- Create all the data base, which in our case comprises 16 columns of data (one column per element), and 50 rows (one per respondent).
- The coefficients tell us the degree to which the respondent would rate that element a 7–9 if the vignette comprised only that element.
- Apply the method of clustering to divide the set of respondents into two groups, and then again into three groups.
- Build a model for each of the two groups, and then build a model for each of the three groups.
- Choose the more parsimonious solution, which is at the same time interpretable.
- Interpretable means that the strongest positive elements ‘tell a coherent story’.
- Parsimonious means that the fewer the number of clusters or mind-sets, the better, so long as the mind-sets tell a story which makes sense.
The results from the clustering suggest three mind-sets, as shown in Table 8. The clustering was done on the coefficients after the ratings were converted to the ‘predicted violence scale’ (ratings of 7–9 converted to 100, ratings of 1–6 converted to 0). The mind-sets are named according to the elements which generate the highest coefficients for the mind-set.
Table 8. Parameters of the model for the mind-sets relating the presence / absence of the 16 elements to predicted violence (Ratings of 7–9 converted to 100), and to predicted happiness (Ratings of 1–3 converted to 100.) The mind-sets were generated based upon the predicted violence scale.
|
|
Mind-Set: 1 No Specific Warning |
Mind-Set: 2 Sensitive to the Economy |
Mind-Set: 3 Family has Problems |
|
Mind-Set: 1 No Specific Warning |
Mind-Set: 2 Sensitive to the Economy |
Mind-Set: 3 Family has Problems |
|
|
Violence |
|
Happiness |
||||
|
Additive constant |
47 |
20 |
16 |
|
20 |
2 |
14 |
C4 |
The husband is sad and depressed |
-4 |
4 |
16 |
|
-10 |
3 |
-2 |
C2 |
The lady is having problems with finances |
-16 |
-6 |
13 |
|
3 |
5 |
-2 |
C3 |
The husband is having job troubles |
-15 |
2 |
13 |
|
-6 |
5 |
-7 |
B1 |
Companies are firing employees |
-14 |
21 |
7 |
|
7 |
0 |
3 |
B3 |
It’s in middle of winter … Christmas |
-6 |
21 |
-7 |
|
18 |
-3 |
12 |
B2 |
Companies are hiring but people working long hours |
-13 |
17 |
-8 |
|
9 |
2 |
5 |
A1 |
The local economy is stressed and in recession |
-11 |
13 |
10 |
|
-5 |
5 |
-7 |
B4 |
It’s summer time |
-11 |
11 |
-11 |
|
16 |
7 |
8 |
D2 |
The family all eat at different times |
7 |
1 |
-9 |
|
-6 |
3 |
-5 |
D3 |
The wife wants to talk but the husband does not |
6 |
7 |
-3 |
|
-7 |
0 |
-5 |
D4 |
The husband wants to talk but the wife does not |
2 |
-6 |
1 |
|
-3 |
7 |
-7 |
D1 |
The family time is shorter together |
1 |
0 |
-1 |
|
-5 |
3 |
-4 |
A3 |
The children are having problems |
-9 |
4 |
7 |
|
-2 |
6 |
0 |
A2 |
The local economy is growing |
-10 |
-2 |
-3 |
|
1 |
5 |
9 |
A4 |
The couple are having long term problems |
-11 |
5 |
9 |
|
-10 |
7 |
2 |
C1 |
The lady starts searching for a job to help out |
-21 |
-16 |
1 |
|
4 |
6 |
1 |
When we look at response times for the three mind-sets (Table 9) we see dramatic differences in the pattern of elements which ‘engage,’ i.e., operationally defined as generating a response time of 1.5 seconds or longer. The elements which drive the segmentation also appear to strongly engage only respondents in Mind-Set 3 (family has problems),
Table 9. Response times for the 16 elements, estimated from the separate models, one for each of the three mind-sets.
|
|
Mind-Set: 1 |
Mind-Set: 2 Sensitive to the Economy |
Mind-Set: 3 Family has Problems |
B2 |
Companies are hiring but people working long hours |
1.2 |
1.1 |
2.1 |
A4 |
The couple are having long term problems |
1.0 |
1.6 |
1.8 |
C2 |
The lady is having problems with finances |
1.4 |
1.4 |
1.7 |
D4 |
The husband wants to talk but the wife does not |
1.3 |
1.5 |
1.6 |
B1 |
Companies are firing employees |
1.0 |
1.2 |
1.5 |
B3 |
It’s in middle of winter … Christmas |
1.2 |
1.2 |
1.5 |
D3 |
The wife wants to talk but the husband does not |
1.0 |
1.7 |
1.1 |
D2 |
The family all eat at different times |
1.2 |
1.7 |
0.9 |
A1 |
The local economy is stressed and in recession |
1.0 |
1.6 |
1.4 |
A2 |
The local economy is growing |
1.2 |
1.5 |
1.0 |
C4 |
The husband is sad and depressed |
1.4 |
1.1 |
1.0 |
C3 |
The husband is having job troubles |
1.4 |
0.8 |
0.6 |
D1 |
The family time is shorter together |
1.0 |
1.1 |
1.4 |
C1 |
The lady starts searching for a job to help out |
1.0 |
1.0 |
1.4 |
B4 |
It’s summer time |
1.0 |
0.5 |
1.3 |
A3 |
The children are having problems |
0.4 |
1.3 |
1.3 |
Mind-Set 3 (Family has problems)
Companies are hiring but people working long hours
The couple are having long term problems
The lady is having problems with finances
The husband wants to talk but the wife does not
Companies are firing employees
It’s in middle of winter are Christmas
Mind-Set 2 (Sensitive to the economy)
The wife wants to talk but the husband does not
The family all eat at different times
The couple are having long term problems
The local economy is stressed and in recession
The husband wants to talk but the wife does not
The local economy is growing
Mind-Set 1 (No specific warning)
No element engages
The nature of people – optimistic versus pessimistic
The original focus of this paper was the pattern of responses of people to vignettes describing a couple who are in a stressful situation. The pattern of responses of our 50 respondents can also show us whether the respondents themselves are typically optimistic, pessimistic, or neither. The analysis is straightforward. We have 24 samples of the respondent’s evaluations of vignettes, with all elements (answers) appearing an equal number of times, and the basic experimental design structure maintained.
In our preparation for modeling we created binary two scales, each 0/100. Each respondent generates an average on each binary scale. When we look at predicted violence, for example, an average of 100 means that 100% of the time, i.e., for all 24 vignettes, the respondent predicts violence will occur. In contrast, if the average if 50, then the respondent predicts that violence will occur on in half the vignettes.
With this way of plotting the data we can look at the respondents, either one at a time or for key subgroups, to determine where the respondent lies on the scatterplot, and what that implies about the respondent. Figure 3 shows the scatterplots for total panel, gender, age, and mind-set, respectively.
Figure 3. Scatterplot of binary transformed ratings. Each point is the average binary rating for a respondent. The abscissa is the average for the respondent for ‘predicted violence’ (rating 7–9 converted to 100). The ordinate is the average for the respondent for ‘predicted happiness’ (rating of 1–3 converted to 100).
The key things to note are:
- The 45-degree line means that that the respondent is neither pessimistic nor optimistic but predicts violence and predicts happiness an equal number of times.
- The further out on the abscissa and the ordinate the respondent falls, the more the respondent is judgmental. There respondent either rates the vignette as describing a situation ending in violence, or describing a situation ending in happiness.
- The closer the respondent falls to 0,0 the less frequently the respondent is judgmental.
- Respondents falling to the right of the line and high on the abscissa (far right) tend to predict violence
- Respondents falling above the line, and high on the abscissa (far up) tend to predict happiness.
- Figure 3 immediately shows the greater negativity of females versus males, Age 50+ versus younger respondents, and Mind-Set 2 versus the other two mind-sets, respectively.
Finding the mind-sets in the population (Attila)
The conventional way to discover different groups in the population is through surveys. When one ‘knows’ the subgroup to which a person belongs, e.g., our mind-sets, it is only nature to believe that there are correlates of membership in the population. If only we could discover those correlates, goes the standard plaint. The ingoing assumption is that people who ‘think similarly’ (our mind-sets) should BE similar on the factors used to measure them. An example is age, another is gender, both of which, of course, are surrogates for various life situations and experiences.
Table 10 suggests that if we are to look to age and to gender as co-variates of segment membership, we are likely to be disappointed. Certainly, as our data suggest, these subgroups exhibit their own general patterns, different from each other, but not suggesting profound differences. In contrast, mind-set segmentation of the type performed with Mind-Genomics data divides people by how they respond, and thus think, in a particular situation.
Table 10. Distribution of the respondents by both the mind-sets (columns) and the more traditional divisions (gender, age, respectively).
|
Mind-Set: 1 |
Mind-Set: 2 Sensitive to the Economy |
Mind-Set: 3 Family has Problems |
Total |
Total |
15 |
17 |
18 |
50 |
Gender |
|
|
|
|
Male |
9 |
7 |
8 |
24 |
Female |
6 |
10 |
10 |
26 |
Age |
|
|
|
|
19–29 |
6 |
4 |
2 |
12 |
30–49 |
6 |
7 |
2 |
15 |
50+ |
3 |
6 |
13 |
22 |
No Answer |
|
|
1 |
1 |
The specificity of the mind-set segments to the test stimuli means that we need a way to assign NEW people to one of the three mind-sets. The system must respect the fact that the mind-sets emerged from the elements specific to this topic and this study. Thus, we end up assigning new people to mind-sets based upon a system which is specific to the study. To this end, author Gere has created a PVI, personal viewpoint identifier which uses the pattern of coefficients from the averages for the three segments. The PVI is created by adding ‘noise’ to the basic summary data for the three mind-sets, and then using them to predict mind-set membership. The six strongest predictors in the ‘face of natural noise in data’ are selected as the cohort to be used to assign new people to one of the three mind-sets. Figure 4 shows the PVI for this study, and the three feedback pages which emerge, depending upon the mind-set to which the respondent is assigned. The feedback pages can be used for further scientific study, for clinical purposes, and even for digital and personal marketing. As of this writing (December 2019) the PVI is available this website:
Prediction of Violence: Violence: http://162.243.165.37:3838/TT20/
Prediction of Happiness: http://162.243.165.37:3838/TT21/
Figure 4. The PVI (personal viewpoint identifier) for the spousal violence study, by which new people can be assigned to one of the three mind-sets uncovered in the research.
Discussion and Conclusions
This paper presents the emerging science of Mind Genomics as a way to bridge the gap between the impersonal, quantitative dimension of social science and the qualitative, story-telling, emotion-filled and narrative-rich material provided by qualitative methods, story-telling, and literature.
The scientific literature dealing with marital violence provides us with a sense of the many different contributors to the violence in the home, mainly between spouses and but directed to other members of the family. There is a body of sociological and psychological data looking for correlates of family violence. The range of these correlated variables is extensive, as can be sensed from the small sample the literature cited here.
The problem with studying violence and other factors of the ‘human condition’ is the virtual impossibility of doing experiments. The ethics of science and the moral responsibility of people to act ethically precludes doing experiments. We are left with observations and reports. Mind Genomics steps in with an attempt to go one step further, using the ordinary individual as an observer of a reported situation (the experiment), and reacting in terms of a prediction of the outcome (violence, nothing, happiness, respectively). In this respect we might consider Mind Genomics in these situations to be analogous to the behavioral economics tool of ‘predictive markets,’ or better ‘information markets’ which uses subjective perceptions embedded in a stock market-like game to drive deep insights into the reasons behind choice [16, 17, 18].
The future holds the promise of learning such as we obtained here, not only for violence in the home, but literally for the many dozens, if not hundreds of life situations that do not permit of an experiment, but may yield some of their secrets to Mind Genomics, which combines the rigor of quantitative science with the richness of cognitively meaningful stimuli actually descriptive of normally lived lives.
Acknowledgments
Attila Gere thanks the support of Premium Postdoctoral Research Program of the Hungarian Academy of Sciences.
The authors wish to thank Dr. Gillie Gabay for her help in formulating the problem and placing it into its academic perspective.
References
- Makepeace JM (1981) Courtship violence among college students. Family relations 97–102.
- Poole C, Rietschlin J (2012) Intimate partner victimization among adults aged 60 and older: an analysis of the 1999 and 2004 General Social Survey. Journal of Elder Abuse & Neglect 24: 120–137. [Crossref]
- Macmillan R, Gartner R (1999) When she brings home the bacon: Labor-force participation and the risk of spousal violence against women. Journal of Marriage and the Family 947–958.
- Miller BA, Downs WR, Gondoli DM (1989) Spousal violence among alcoholic women as compared to a random household sample of women. Journal of Studies on Alcohol 50: 533–540. [Crossref]
- Brinkerhoff MB, Grandin E, Lupri E (1992) Religious involvement and spousal violence: The Canadian case. Journal for the Scientific Study of Religion 15–31.
- Dobash RP, Dobash RE, Wilson M, Daly M (1992) The myth of sexual symmetry in marital violence. Social problems 39: 71–91.
- Fyfe JJ, Klinger DA, Flavin JM (1997) Differential police treatment of male‐on‐female spousal violence. Criminology 35: 455–473.
- Stith SM, Farley SC (1993) A predictive model of male spousal violence. Journal of Family Violence 8: 183–201.
- Moskowitz HR (2012) Mind genomics’: The experimental, inductive science of the ordinary, and its application to aspects of food and feeding’. Physiology & behavior 107: 606–613. [Crossref]
- Kahneman D (2011) Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.
- Blumenthal AL, Danziger K (2001) Wilhelm Wundt in History: The Making of a Scientific Psychology. Springer Science & Business Media.
- Nichols KA, Champness BG (1971) Eye gaze and the GSR. Journal of Experimental Social Psychology 7: 623–626.
- Stipp H (2015) The Evolution of neuromarketing research: From novelty to mainstream: How neuro research tools improve our knowledge about Advertising. Journal of Advertising Research 55: 120–122.
- Nosek BA, Greenwald AG, Banaji MR (2005) Understanding and using the Implicit Association Test: II. Method variables and construct validity. Personality and Social Psychology Bulletin 31: 166–180. [Crossref]
- De Hoon MJ, Imoto S, Nolan J, Miyano S (2004) Open source clustering software. Bioinformatics 20: 1453–1454.
- Abramowicz M (2004) Information markets, administration decision making, and predictive cost-benefit analysis. The university of Chicago Law Review 71: 933–1020.
- Box GEP, Hunter WP, Hunter JS (1978) Statistics for Experimenters, New York, John Wiley.
- Gofman A, Moskowitz H (2010) Isomorphic permuted experimental designs and their application in conjoint analysis. Journal of Sensory Studies 25: 127–145.