Data Analysis Project II

Data Analysis Project II 

Vanessa Ralph 
 
Description 
After collecting interview data via the notability application on the iPad that replays the 
conversation between the researcher and student while tracking students’ sketches, the data 
were analyzed to account for both the audio and visual manifestations of students’ 
problem-solving processes. A graphic that articulates the various perspectives and 
methodologies used to code and analyzes the data is provided in Figure 1 below. This process 
involves the researcher playing back the interview recording at least three times following its 
transcription and provides an in-depth review of the data collected. 

 
Figure 1​. Methodological approach. 
 

First, each students’ interview was organized into a series of stages mapped onto the 
steps along a hypothesized solution path for the solving the stoichiometry prompt provided to 
the student during the interview (denoted as grey rectangles in the figure above). The beginning 
and end of the dialogue related to each step were identified using process coding. Organizing 
the interview along the hypothesized solution path allowed the researchers to analyze discrete 
segments that serve as microcosms of the students’ expressed challenges, the questions asked 
by the researcher to engage the student about the challenge, and their successes with regard to 
arriving at a solution. The hypothesized solution mapped used during process coding to identify 
dialogue related to each step is depicted below in Figure 2. 
 

 
Figure 2​. The solution map used for the stoichiometry item in the protocol. 

 
Then, rounds of causation coding (denoted as ➡ in Figure 1) through the data were used 
to identify shifts in areas of difficulty and the bidirectional dialogic exchange that allowed the 
student to progress. These areas in which the student struggled to express or model how to 
proceed with their solution were analyzed using three different, second cycle coding 
approaches: 1) A deductive coding scheme designed in the author’s prior work (Ralph ​et al.​ , 
2018b; see Table 1) was used to identify students’ observed difficulties that aligned with a priori 
expectations as informed by the literature base cited in the article, 2) any emergent and 
meaningful codes not encapsulated by the deductive coding scheme were coded inductively 
using the student’s averments (e.g. In Vivo coding), and 3) via illustrated analytic memos drawn 
by the first author used to envision an interpreted internal visuosketchpad or cognitive map 
before each dialogic exchange flagged via causation coding. Each of these tools were used to 
assess and provided greater detail as to what was known of students’ challenges with 
stoichiometry, emergent challenges un- or underexplored in previous scholarly works, and a 
nuanced visual analyses of the researcher’s interpretation of their learning progression in 
connecting what they’ve learned in lecture to the prompt.  
 
Table 1​. Deductive coding scheme for the stoichiometry assessment items 

Theme  Category 

Systematic  Conversion Factors between Units of Measure (Mass to Moles; Moles to 
Proportional  Mass) 
Reasoning (S) 
Setting up Mole Ratios (Mole to Mole Conversions) 

Logical Arrangement of Steps Resulting in Appropriate Units 

Matching an Elemental Symbol with its Atomic Mass (Elemental Symbols 
and the Periodic Table) 

Representational  Mole Ratios as Conversion Factors between Chemical Species (Moles to 
Competency (R)  Moles; Subscripts and Coefficients) 

Using the Chemical Formula to Calculate Formula Mass (Subscripts) 

Conceptual  Diatomic Molecules vs. Elements (H​2​ vs. H and O​2​ vs. O) 
Mass is Conserved 

Understanding (C)  “Mass” vs. “Moles” 

“Molar/Formula Mass” vs. “Mass” 

“Excess” Reagents are not Limiting 

 
Finally, a third cycle coding approach was then enacted, known as pattern coding, on the 

codes and visual memos to devise categories and themes comprising the evidence-base 
collected as pertinent to the research questions regarding the visualization of molecules and 
the proportions by which they react. Ultimately, elements of ethnography and arts-based 
research were used to develop theoretical constructs observed for students linking 
representational competency and proportional reasoning and the challenges observed by the 
participating students. No studies, to date, have contributed to a theoretical framework 
regarding the connection students make in visualizing the interactions between particles and 
the proportional relationships by which they interact (particularly for those of low math 
aptitude). Thus, a detailed account of these students and their interactions with the researchers 
regarding contributes a novel source of information that could be used to inform intervention, 
instruction, and assessment design.  

In the interest of reflexivity, and thereby transparency, the researcher subscribes to 
Gardner’s theory of multiple intelligences and observes that while students of low math aptitude 
struggle disproportionately with assessment items in chemistry, if provided a means to 
communicate that is not alphanumeric, their likelihood to succeed may be better supported. 
This emphasis on visual aids is congruent with an appreciation for what visuals provide as an 
alternative means of communication. This is the rationale behind arts-based research and one 
adopted by the researcher to better serve as the instrument of measure in this qualitative study. 
These individual student progressions as documented as visual analytic memos were then 
compared between the students' interviews to generate hypotheses as to the challenges 
particular types of assessment items could elicit in students of low math aptitude. These 
features can then be measured quantitatively in future studies of which the scope would include 
comparing the performance of students of variable math aptitude score on assessment items 
that are representative of the types identified.  
 
Results 
Each student manifested a variety of challenges associated with interpreting chemical 
equations and using these stoichiometric proportions to arrive at a numerical value for the 
quantity of a product. Dividing these findings into themes, students’ responses demonstrated a 
variety of misconceptions along representational, conceptual, and algorithmic realms of the 
topic. 
 
Representational Competency 
In translating a chemical equation to the more pictorial submicroscopic diagram, more nuance 
is achieved. The most striking difference between these two forms of representation (see Figure 
3 for a model of each) is how bonded atoms and the phase states of compounds are 

communicated. In a chemical equation (the alphanumeric representation provided in Figure 3a), 
stoichiometric proportions - or the ratio by which these species react - are communicated with 
the coefficients preceding each chemical formula. The chemical formulae of each species 
follow the coefficient in which each capital letter marks the start of a new element which can be 
represented using one- or two-letter abbreviations. Subscripts in the chemical formula are 
meant to note how many of each element comprises the compound. No differences in a 
chemical formula are found for the various phase states a chemical compound occupies. These 
states are instead represented by the italicized ​s, l, aq, a​ nd ​s ​found enclosed in parentheses 
following the chemical formula to identify solid, liquid, aqueous, and solid states of matter, 
respectively.  
 

 
Submicroscopic diagrams tend to present each elemental symbol contained within a 
circle.Overlapping circles are intended to be representative of covalently bound atoms. Atoms 
bound ionically can be represented in one of two ways depending on the state of the compound. 
When using a chemical formula, the charges of ionically bounded elements are not depicted. 
Instead, chemical formulae comprised of elements with both metallic and non-metallic 
characteristics are expected to be known by the reader as an ionic compound in which charged 
atoms hold proximity as a result of electrostatic interactions. These interactions are more 
explicit in submicroscopic diagrams while also reflecting the added nuance of the state these 
compounds occupy. Atoms comprising ionic compounds are typically represented in 
submicroscopic diagrams by denoting each atom’s elemental symbol enclosed in a circle along 

with the charge of its nucleus (demonstrated using superscripts). Solid ionic compounds are 
drawn in close, but not overlapping, proximity to one another to demonstrate the electrostatic 
nature of the bond in concordance with the solid state lattice observed within these solids. 
Aqueous ionic compounds, however, have been dissolved by a solvent (often water) and are 
depicted as diffuse in the diagram (see sulfate or SO​42​ +​ in Figure 3b).  

In both forms of representation, it is often assumed each chemical species is directly 
involved in the chemical reaction. Therefore other chemicals inherently involved in the aqueous 
component of the reaction (e.g. water molecules) but not taking part in the reaction are 
excluded from the balanced chemical equation. Students at this stage in their exploration of 
chemistry, are not expected to know electron or molecular geometries and thus are not 
discounted for portraying polyatomic ions as a single unit. In fact, conversations with the 
instructors suggest they are explicitly told polyatomic ions “stick together” and do not dissolve 
in water. This primary model is often used to simplify the concept of bond energy and 
resonance until later coursework. Figure 4 presents the responses of 3 first-semester, general 
chemistry students to the semi-structured, cognitive interview prompt below: 
 

Consider the following balanced chemical equation: 
 

2 Al (s​ ​) + 3 H​2​SO​4​ (​aq)​ → Al2​ ​(SO​4)​ ​3​ (s​ ​) + 3 H2​ ​ (g​ ​) 
 

1. Draw a picture representing this balance chemical reaction between Al (​s​) and 
H2​ S​ O​4​ (a​ q​): 

 

 

 

The first student (Figure 4a) mirrors the chemical equation with an accurate 

representation of the proportions of reacting chemicals no representation in the differences of 

the states to which they exist. An understanding of these differences with phase states and how 

they are depicted on submicroscopic diagrams was assessed on students’ first interim exam. 

These images were drawn up to one week following their second interim exam. The second 

student (Figure 4b) uses colors to represent the different atoms and does not include elemental 

symbols. The students’ response process to this item is important here in a way that the picture 

does not communicate. Here, it was imperative to have a video recording of the student 

describing the steps and corrections made to the illustration. Starting with two conjoined 

spheres to represent “2 Al” the student quickly identifies (with no prompting) that these atoms 

are not covalently bound.  

 

“Wait, that’s not Al2​ ​ it’s 2 Al”. 
 

The student draws an “X” over this pair of atoms and redraws the diagram showing two 

spheres of aluminium that are not overlapping or covalently bound. The student then sees “H​2”​  
in “3 H​2​SO​4”​ and begins drawing three pairs of two conjoined circles representative of 3 
molecules of hydrogen gas and not two charged hydrogen ions in aqueous solution with a 

sulfate ion. The student then pauses and whispers “SO​4…​ ” as they consider where the 
polyatomic ion fits in with the submicroscopic diagram. Ultimately the student illustrates a blue 

sulfur atom to the right of each pair of grey hydrogen atoms and surrounds the hydrogen with 

four, red oxygen atoms. This representation separates the polyatomic ion in a way that would 

not occur in aqueous solution but still maintains the description of “sticking together”. The 

compound overall is illustrated as either a solid or covalently-bound compound. Seeing “H2​ ​” 
again in the products, the student repeats their representation of 3, diatomic hydrogen gas 

molecules; this time the student is accurate in their representation. The student then draws the 

compound Al2​ (​ SO​4)​ 3​ ​ accurately as a unified, solid compound without using elemental symbols 
nor superscripts denoting the charges of each atom which inaccurately conveys this solid 

compound as covalently bound. 

The third and final student (Figure 4c) presents a submicroscopic diagram with more 

fundamental errors than the prior two. First and foremost, aluminium is represented in this 

drawing using an “A” and not “Al”. Plus signs “+” used in a chemical equation to separate 

reacting species of chemicals are included in this student’s submicroscopic diagram 

suggesting the physical space between these two chemical species in the equation may be 

separated by the abstract force of the plus sign. This is another example of the mirroring that 

occurs when students are asked to translate between these two forms of chemical 

representation. The student proceeds to draw 3 circles around “3 H​2S​ O​4​” thereby depicting 9 
moles of this chemical species. Upon prompting by the interviewer, the student eventually 

realizes this error and corrects it by scratching out the “3” before each chemical formula. This 

student, as did each other student, does not depict “H​2S​ O​4​” as 2 atoms of positively charged 
hydrogen atoms ionically bonded in aqueous solution to a sulfate polyatomic ion (SO​42​ -​) but 
instead sticks keeps the atoms together in a manner that mirrors the chemical equation without 

making the cognitive divide toward the benefits of using a submicroscopic diagram. When 
illustrating the products, this student misattributes the parentheses of “Al​2(​ SO​4​)​3”​ as “(Al2​ ​SO​4)​ ​3​” 
thereby altering the provided molecular proportions and does not illustrate this substance as a 
covalently bound solid. Molecular hydrogen atoms are then illustrated as 3 circles of “H” and not 
“H​2”​ again inaccurately depicting the molecular proportions of the products and depicting a lone 
H atom rather than the diatomic that typically exists as gaseous hydrogen. How these images 
are constructed overtime matters and these screenshots do not accurately depict how the 
students arrived at these responses. This is where the strategy of illustrated analytical memos 
can more accurately demonstrate the challenges faced by students responding to cognitive 
prompts. While including video data in a research article (ala Harry Potter newspaper articles) is, 
to date, not an available option for researchers in communicating their findings. At the very 
least, the stages for changes made by students to their solution response processes is more 
accurate in presenting the data acquired in cognitive elaboration interviews. Illustrative 
analytical memos add elements of narrative and arts-based research by including 
participant-voiced quotations and the exasperations or (very rarely) laughter students express 
when solving chemistry problems along with these stages of students’ response processes 
allowing for the researcher to both explore the implications of visual data and communicate 
these abstract concepts visually to the reader. Figure 5 presents analytical memos created from 
each of the 3 students responses to this cognitive elaboration prompt and quotations relating to 
the changes made to their illustrations.  
 





 
 
Reflection 
In the results, I describe a few benefits of the illustrative analytical memo technique used to 
serve almost as an extension of and amalgamation between narrative and art-based inquiry 
methodologies. Cognitive interviews, often via the think-aloud approach, in discipline-based 
education research is common practice. The data is often either a theme-coded transcript or 
photographs of the students’ solution process communicated via scratch paper. The limitations 
of these modes of collecting the data are that researchers and later, the readers of this 
research, lose the nuance of ordering, correcting, and elaboration by which students construct 
these responses. Images built in stages with the quoted text inherent to the students’ change of 
the diagram can amend the data lost to the chosen method of data analysis. These images are 
what can comprise illustrated analytical memos. I chose to include a bit more illustration in 
these as it helped me to resolve the questions in my mind about the students’ journey through a 
solution process and illustrated response.  

Ultimately, I found the process clarifying and helpful as a researcher but I do worry about 
its interpretation as “robust” or even inaccurate/biased by other researchers. While I’ve learned 
about qualitative research that the researcher is the tool used to analyze the data, I am also 

aware of what researchers in my field often think of how and what constitutes trustworthy 
methodologies and representations of qualitative data. By incorporating what could be 
described as cartoon-like figures illustrating my mental and visual processes of the data, I could 
leave myself open to criticism in a way that may not be prevalent in a figure comprised of 
students’ processes built in stages surrounded by quotes that resulted in these changes to the 
diagrams or solution process. Regardless, I find the latter (stages and quotes) helpful for 
communicating my findings to others in a way that ameliorates some of the data lost by images 
of student solution processes and the former (illustrated analytical memos) a helpful way for 
me to visualize my data and the cognitive processes that may be occurring within the student 
while solving these prompts.