Presentation

I met with a few members of my informal PLC (we meet regularly to "swap" lesson ideas.) We came together this week to download about this year and talk about next year, specifically determining benchmarks and proficiency levels by grade level as well as discussing how to meet the global competency requirements of the World Language Program Review. I also took this opportunity to present the findings from my action research.

We all agreed that we liked using stories in our Spanish classes and we had a good discussion on what kinds of stories work best and how best to present them to our students. None of us have had TPRS training but we all used some of the techniques in our class. As part of my professional growth plan I'd like to learn more in-depth about this method as well as continue my research on its effectiveness in the FLES classroom.

Data analysis - behavioral engagement scores

Analysis of my data proved overwhelming at first but after a few false starts I have waded through. So what did I come up with? 

For the behavioral engagement scores I viewed the videos and scored each student's behavior for every 30 second interval per my rubric. In addition to finding the mean - which showed students on average were engaged in the lessons - I also looked at the min and max for each lesson to see what specifically was going on during those times. I found that the highest levels of engagement came when I was being dramatic in telling the story (using silly voices and sound effects) and when students were asked to be dramatic (pretending to be ill while sing Head, Shoulders, Knees, & Toes.)

Table 1
Behavioral Engagement Scores
	Mean	Min	Max	SD
Storytelling lesson	2.96	2.60	3.40	0.18
TPR & Song lesson	3.08	2.86	3.46	0.18

I ran a paired samples t-Test to see if there was a difference between the two lessons and found that there was no significant difference in the scores for the storytelling lesson (M=2.96, SD=.018) and the TPR & song lesson (M=3.08, SD=0.18) conditions; t(14)=-1.52, p>.05.

Data Analysis - Survey data

I would like to take a moment and profess my continuing love of Excel. I first fell in love as an associate account manager who put together sales reports and audited tens of thousands of lines of data for my accounts (VLookup, pivot tables, conditional formatting, and nestled IF statements!) But I'm falling in love all over again with descriptive statistics, correlations, paired t-tests, and ANOVAs. Now onto the data...I used descriptive statistics as well as looked to see if there were any correlations in my survey answers. Here's what I came up with.

Table 2
Student Survey Responses
	Storytelling Lesson		TPR & Song Lesson
Questions	Mean	SD	Mean	SD
1. I get really involved in classroom activities	4.5	0.64	4.6	0.74
2. I actively participate in classroom discussions	4.2	0.86	4.3	1.10
3. I form new  questions in my mind as I join class activities	3.9	0.92	3.3	1.45
4. I compare things I am learning with things I already know	3.9	0.80	3.8	1.37
5. I work with other students and we learn from each other	4.1	1.03	4.3	0.88
6. I can understand the meaning of the lesson even though it's in Spanish	4.1	1.06	4.3	1.10

There was a strong correlation between the male students’ responses to the survey question “I compare things I am learning with things I already know” (r=.54, p < .05) for the TPR & Songs lesson. There was also a high correlation between male students’ responses to the survey question “I work with other students and we learn from each other” (r=0.51, p < .05) for the TPR & Songs lesson. For both questions the male students in the class responded with Strongly Agree.

I was happy to see that students answered Agree (4) on question 6 "I can understand the meaning of the lesson even though it's in Spanish" for both lessons. I work hard to make sure that I'm giving students comprehensible input or Krashen's i+1. My literature review suggests that students who cannot understand are much more likely to become behaviorally disaffected so in both cases students reported that they understood what was going on during class.

Data Collection - Thick Description of Qualitative Data

I am still going through the two videos of my mini-lessons and using constant comparative coding. The first part of that means watching the videos and writing notes on what I observe. Below are two "thick" descriptions of my observations from the mini-lesson where I told the students the story of Caperucita Roja or Little Red Riding Hood.


Raw data:

Student 10 - repeats after me without being asked "Sr. Lobo!" Says "Yes!" when hunter is introduced. Raises hand and answer question ¿Cómo está el lobo? with "Tengo MUCHA hambre." Having fun.

Student 6 - does not maintain eye contact, instead picks and chews on nails. Shy ELL student. Looks bored.

Student 1 - acts out parts of the story - when Sr. Lobo gets an idea he pops up 2 fingers by face and opens mouth and eyes wide. Mimes large teeth and growls during discussion about how CP knows the Sr. Lobo is not her abuela. Doesn't smile much but still appears to be enjoying the story.

Thick description:

At almost 4 minutes into the lesson, I introduce Señor Lobo. Student 10, an African-American female who is a high performer in Spanish class as well as in other classes , repeats after me without being asked. Several students turn around and look at her. She smiles and does not seem embarrassed. Later, she exclaims, "Yes!" when I introduce the hunter to the story. She is quick to raise her hand and answer questions. She answers the question ¿Cómo está el lobo? with "Tengo MUCHA hambre." She is very engaged throughout the story and appears to be enjoying herself.
  
Student 6, an Asian student whose first language is Chinese, sits quietly throughout the lesson. he does not appear to be very engaged with the story. She does not raise her hand to answer questions. She is shy and speaks softly when she does speak. She does not maintain eye contact throughout the story, instead chewing on her nails. She looks up during the more exciting parts of the story and smiles but it's possible as an ELL student that she is accustomed to tuning out longer stretches where she has to listen for meaning. It's also possible she is not as familiar with the story of Little Red Riding Hood as her classmates. 

Student 1, a white male does not smile much but he sits up and looks at the teacher throughout the mini-lesson. At certain points he acts out parts of the story silently to himself. When Sr. Lobo says, "Tengo un idea!" he pops up his two index fingers on either side of his face and opens his mouth and eyes wide as though he has had an epiphany similar to the wolf in the story. Later when another student answers that Caperucita Roja knows the wolf is not her grandmother because of its big teeth he mimes having big and sharp teeth and growls quietly.

Data Collection - Surveys

Below are the results of the surveys my students took following each lesson.

	TPR & Songs
Student Number	Q1	Q2	Q3	Q4	Q5	Q6
1	5	4	2	5	5	5
2	5	5	3	4	4	5
3	5	5	3	4	3	3
4	5	5	5	4	4	5
5	5	5	3	4	4	5
6	4	3	1	3	3	4
7	3	4	1	1	5	4
8	5	4	5	5	5	5
9	4	4	3	3	3	4
10	5	4	5	5	5	5
11	5	5	5	5	5	5
12	3	1	3	1	5	1
13	5	5	2	3	3	4
14	5	5	5	5	5	5
15	5	5	4	5	5	4
	69	64	50	57	64	64

	Storytelling
Student Number	Q1	Q2	Q3	Q4	Q5	Q6
1	4	3	3	4	5	5
2	4	5	3	3	4	5
3	5	5	5	3	1	3
4	5	5	4	4	5	5
5	4	5	3	3	4	5
6	5	3	4	3	3	3
7	3	3	2	5	5	2
8	5	3	3	5	4	5
9	4	4	4	4	4	4
10	5	4	5	5	5	5
11	4	4	4	4	4	4
12	5	5	4	3	4	3
13	4	5	4	4	4	3
14	5	5	5	5	5	5
15	5	4	5	4	4	5
	67	63	58	59	61	62

Right off the bat I see a difference between question 3 - "I form new questions in my mind as I join in class activities." I haven't performed any descriptive statistics to see how these actually compare but I do wonder if the difference comes from the method of teaching - in the storytelling lesson students had to sit and really figure out what was going on while in the TPR and songs they were much more active and in the moment. Just my thoughts at the moment. The rest of the results seem pretty close so I will be interested to see what my analysis will bring to light.

Methods

Procedures

            Since this action research study aims to measure students’ behavioral engagement, I will rely on a mixed methods approach, using classroom observations and student surveys. Seeing that I cannot teach and observe at the same time I will videotape my lessons with my focus class. As I am testing to see if storytelling engages more students than TPR and songs, I will teach and videotape two separate lessons with the same class. For the control lesson, I will use more traditional methods and introduce body parts using gestures and a song. In the experimental lesson, I will use the familiar story of Little Red Riding Hood using the same target vocabulary. In both lessons, students will be encouraged to participate by answering questions, talking with partners, and either singing or acting out the story.

Since feelings of competence influence how students engage behaviorally during classroom instruction, I will have students will fill out a short Likert item survey after each lesson to determine how they felt about their comprehension and their participation during the lesson.

Measures

            Teachers and researchers agree that there are many potential benefits to student engagement, but accurate measurement is necessary (Carter, et al., 2012). With that in mind, I will review the videos of the two different lessons and describe each students’ behavior during the instruction. This qualitative data will then be coded and analyzed for specific patterns, using constant comparative coding.

In addition to describing the student behavior in the videos, I will also try to quantify their behavior using interaction analysis. The study, “Scaffolding for student engagement in Elementary School Reading Instruction” (2006) conducted by Lutz, Guthrie, and Davis also tried to measure student engagement in class. In this study, the researchers took efforts to decide what engagement looked like before watching the videos of instruction. They created a protocol whereby they assigned a point value to each reaction (Lutz, Guthrie, and Davis, 2006).    Malloy, Parsons, and Parsons (2013) took this instrument and adapted it for their study on student engagement (appendix A). Students’ body language and their behavior during class express unconscious thoughts (Marshall, 2010). A kinesic analysis of their body language and participation in class will allow me to see which types of activities engage their interest.  By using the preset rubric with the different reactions will allow me as their teacher to assess my students’ behavior in a more neutral manner.  I will view the videos and score each students’ behavior according to the rubric in appendix A every 30 seconds. This microanalysis of their behavior will give me a more detailed picture.  The scores will be tallied and using descriptive statistics, I will analyze the results.

Immediately after each lesson, students will also fill out a short survey with Likert items on how they performed during class that day. These questions come from the Classroom Engagement Inventory devised by Ze, Bergin, and Bergin (2014) for their study “Measuring Engagement in Fourth to Twelfth Grade Classrooms: The Classroom Engagement Inventory” (appendix B). Since this study focuses primarily on behavioral engagement I have only included those questions relating to behaviors in class. I discarded questions that relate to affective and cognitive engagement from the survey and I added one additional question about how much they comprehended what was going on during class since I conduct classes mostly in Spanish with very little English. Combined with the nonverbal analysis of the students’ behavior during class, these surveys will allow students to express their feelings regarding the lessons and how well they paid attention and focused during the class.

References

Carter, C. P., Reschly, A. L., Lovelace, M. D., Appleton, J. J., & Thompson, D. (2012). Measuring Student Engagement among Elementary Students: Pilot of the Student Engagement Instrument--Elementary Version. School Psychology Quarterly, 27(2), 61-73.

Lutz, S. L., Guthrie, J. T., & Davis, M. H. (2006). Scaffolding for Engagement in Elementary School Reading Instruction. Journal Of Educational Research, 100(1), 3-20.

Malloy, J. A., Parsons, S. A., & Parsons, A. W. (2013). Methods for Evaluating Literacy Engagement as a Fluid Construct. Literacy Research Association Yearbook, 62124-139.

Marshall, C. (2010). Data collection methods. In Designing qualitative research (pp. 97–150). Thousand Oaks, CA: Sage Publications. Retrieved from http://www.sagepub.com/upm-data/10985_Chapter_4.pdf

Ze, W., Bergin, C., & Bergin, D. A. (2014). Measuring Engagement in Fourth to Twelfth Grade Classrooms: The Classroom Engagement Inventory. School Psychology Quarterly, 29(4), 517-535. doi:10.1037/spq0000050

Appendix A: Behavioral Engagement Rating Scale (Taken from Malloy, Parsons, & Parsons, 2013)

Rating	Description	Behavior
1	Clearly not engaged	Sighs, looks bored, yawns, head down, distracted by something unrelated to the task, not participating, not paying attention, off task
2	Difficult to tell	Bland expression, monotone, not off task but not particularly involved, wavering attention to teacher/classmate/task, flipping pages without looking, attention but partial
3	Engaged	Maintains attention, appears interested, clearly on task, posture toward speaker (does this for an entire minute); other evidence: writing, speaking, clearly listening, brief response
4	Highly engaged	Posture or tone reflects enthusiasm or excitement, eagerness to participate, response reveals deep or critical thinking, makes connections, response is extensive, elaborates

Appendix B: Behavioral Engagement Survey Questions (Adapted from Ze, Bergin, & Bergin, 2014).

Question	1-Strongly disagree	2-Disagree	3-Neutral	4-Agree	5-Strongly Agree
I get really involved in class activities.
I actively participate in class discussions.
I form new questions in my mind as I join in class activities.
I compare things I am learning with things I already knew.
I work with other students and we learn from each other.
I can understand the meaning of the lesson even though it’s in Spanish.