Meeting Notes, 6/2/04

Possible Design/Learning "Variables"/Elements.

Child Development
Metacognitive Support with Technology
Researchers in Children's use of technology and interaction
Kirsten Risder, Libby Hanna
Usability Testing with Children
Teachers as Mediators of student knowledge.
Teachers conduct a thorough analysis (of classroom? Good and Brophy, teacher research, modelling the character of the classroom as active learning?)
Meeting the standards in primary science

Design discussions should be grounded in theory

Xiao's video analysis process:
Add to field notes,
Note interesting things viewed in video missed during observation,
Theoretical notes.

Article Review: Turner & Meyer, 2000

Turner, J.C. & Meyer, D.K. (2000). Studying and understanding the instructional contexts of classrooms: Using our past to forge our future. Educational Psychologist, 35(2), 69-85.

They mainly focused on academic instruction and motivation to learn. Similar to Flanders who they cite. Discuss different methods used to study context: observation, interview, discourse analysis.

The study of classroom context requires investigating more than one variable at a time, qualitative inductive component, attempt to answer How and Why questions, that researcher be present in classroom. They propose more multimethod approaches: methodologies, designs and descriptions. They desire more context in theory-building. Need for explicit definitions of the parts of classroom being investigated.

Chapter Review: Huffman, 2002

Huffman, D. (2002). Evaluating science inquiry: A mixed-method approach. in Evaluation of Science and Technology Education at the Dawn of the New Millennium, edited by Altschuld, J.W. & Kumar, D.D. (Eds.). Kluwer Academic: New York.

Interesting theoretical framework for inquiry. Not sure if they made the case for mixed methods. Didn't see a need and purpose that called for MM. Maybe a correlation to the research question.

Article Review: Parke & Coble, 1997

Parke, H.M., Coble, C.R. (1997). Teachers designing curriculum as professional development: A model for transformational science teaching. Journal of Research in Science Teaching, 34, 773-789.

I felt like they needed to "measure" process not product. Good Lit Review. They used a similar treatment to what we will do next year.

Data Management memo

3/16/04

Type of Data: Storage Location and Procedure
Hand-written field notes: Filed in Cabinet, once they are typed up uploaded to Server, save in Data directory, files names are descriptive
Printouts of Word Files: Filing cabinet, in folder of teacher
Audio Tapes: Filed in Cabinet, for permanent storage until destruction, may be digitized later on for testing purposes
Wav files: ?
Transcripts, Word: Uploaded to Server
NVIVO project files: Newest data set will be uploaded to Server
Transana project files, ?
Video Tapes, Filed in Cabinet, for permanent storage until destruction
Student work, Field in Cabinet, copies of all work related to field notes.
Lesson Plans, Word, Uploaded to Server, save in Data directory, labeled descriptively
Email, ?

Notes:
·I need a tape player with line out and record into sound card
·I need sound software to record to wav from line in
·A possibility is to store all wav files to CD until later analysis and transcription

Audio Method notes

3/10/04

1.Download audio(observation, interview) in Sony Voice Editor 2 (proprietary for that particular digital recorder)
a.I used the internal microphone that worked well.
b.I saved it to 11Mhz wav file, monoaural, it was a smaller file size and sounds just as good.
2.Burned it to a CD at home
a.The computer in my office didn’t read the CD because the CD drive is too old.

Video and Audio Methods: Video Testing

2/23/04

I spent a number of hours investigating DV and DVD options. First I realized that my own camcorder is not backwards compatible with Hi8 tape. I borrowed a digital8 from Brenda. So I was able convert the analog tape to digital. That’s good in the cases where there are any digital camcorders around. Currently the department has 1 Digital8 camcorder and 2 Hi8 camcorders. Which is probably enough, although recording in analog is not the best.
·I’m looking at the Sony Digital8 TRV460 at $400 a piece, it’s got 20x optical zoom and is backwards compatible.
·I looked into freeware options, but I think it’s too complicated and not worth my time learning right now. I spent some time looking into Mac options. iDVD is available as a separate package called iLife which costs $50. I think the G5 with Final Cut Express and iDVD is the solution we are looking, but I don’t know if that’s going to do what I want and at the quality that I’m looking for.
·I just realized that the Hi8 camcorders don’t have mic inputs, so they are useless to us.
·The Microsoft DV avi output from the analog tape was pretty good, but a little pixilated along the edges. It’s the best quality test so far.
·I looked into shotgun mics, a simple Audio-technica costs $60 with the kind of quality I think we are looking for. I’m looking for such mic’s that have a camcorder attachment kit like offered from Sony. It’ll be easier if it’s all one unit.
·I looked into developing in Flash but I don’t think it’s really made for what we are trying to accomplish.
·Nan, Fawzia and Tara are interested in learning Premiere in much greater depth. I think they’ve all taken the Premiere I Star class. They are novices. I’d like to offer them a chance to work with video in a real context. They can do the video processing and exporting. I build a community of practice.
·I introduced Lin to the camcorder. She picked things up quickly. It took about thirty minutes of playing with the camcorder. I think she’ll be fine when it comes time for data collection although I worry still.

Video and Audio Methods: Video Conversion Test

2/18/04

On Monday I tested out compiling a video file on Premiere on the main projection computer. Here’s what I found:
·Output video quality was very poor for Quicktime.
·Downloading raw video to computer takes a lot of time.
·Rendering a timeline for video file takes a lot of time.
·I could not get the audio tracks to play separated. They play simultaneously. I don’t know how to do this with Premiere, if it’s possible at all.
·I could not get on the computer until class was over, about 5pm.
·I tried outputting to MPEG, that was better I think.
·I tried playing my Mpeg on my computer, it didn’t work.
·I played the DVD version, m2v extension, there was no sound.
·The VCD and the SVCD were decent quality with a little pixelation. Those were the best I could generate from this round of testing.
·I found Premiere pretty easy to use, although I felt that there were definitely more expert ways of downloading video to the computer. I guess I could have tried StudioDV, but that crashes all the time.
On Wednesday, I’m trying out the PowerBook G4 laptop with iMovie.
·Downloading is in real time it took forever.
·iMovie doesn’t let me do the “enable tracks” only the Quicktime Player full version allows you to do that.
·I will try playing this on my laptop at home.

Video and Audio Methods: Next Steps

2/13/04

Test video-audio compilation in Adobe premiere (Test this weekend?)
Produce Quicktime movie with multiple audio tracks.

Contact Raul Zaritsky
What are the hardware minimums for good quality?
What are the common mistakes?
What kinds of research questions is it good at answering? Not good at?

Practice test in the classroom
audio mic-ing with teacher (inconvenience, student distraction)
digital video on tripod (power cords, student distraction)
practice doing a reflection (audio quality, equipment use, teacher gets used to it)

Video and Audio Methods: Analysis Phase

2/5/04

The first step is to download to digital files to a computer, any computer. Then to transfer them to a Data Repository on Server? The quick solution once we get up and running is to buy external Firewire drives for about $200, that will allow us to at least save the data. Although we have to spend the time analyzing the data. Looking back the researcher should mark times in their own field notes, so that they can be referenced in the video segments. Even doing so during the reflection.

(Looking back, I wish we had enough knowledge and money to buy everyone a mac laptop to download and rip their own tapes as soon as it was recorded. It has probably taken me over 200 manhours to convert these tapes into QT movies, after the fact.)

Video and Audio Methods: Data Collection Phase

2/5/04

The checklist of equipment: Digital Camcorder, 10’ A/V cable ($15 @ Radioshack), Extension cords, Tripod ($30 @ Radioshack), Extra tapes, Make sure battery is charged just in case, Digital Recorder, lapel microphone, spare batteries AAA and button, DC power source?, Laptop? to download audio after class especially for 1.5 hour length classes. The Digital recorder is best used at SP mode, 2.5 hours capacity. LP is lower quality but probably okay for the reflection where the quality is not difficult.

Action Item: A full test run should be conducted by the end of February. Items to check on: classroom coverage angle, setting up with tripod before class starts, access to outlets, effect on classroom/student behavior, How much does the teacher go off screen (sampling), Any unforeseen technical problems occur, Learning phase by research team on using all of this technology. Make sure all students have submitted approval forms. Tapes are up to 60 minutes. For an 1.5 hr class, the tape should be changed at an appropriate classroom break from 50 to 60 minutes. Perhaps this can be arranged with the teacher. Discuss with the teacher if they are doing an outside activity. Make sure that the teacher has a television in the room.

Mic-ing up the teacher, testing to see if we get the kind of data we are looking for, Testing different mic sensitivities, Does the mic affect students? Does the mic affect the teacher? Is the audio afterwards usable? Can the researcher do this without difficulties? Can they troubleshoot, if not what contingencies are setup?

How convenient is it for the teacher to do the reflection afterwards? Can they do it? Will they do it? Do they want to do it? Do they find it useful at all? Do they mind doing it? Effect on the researcher, that’s a lot of data collection! Are we getting the type of data we are looking for? Are we getting data that’s serendipitous?

Video and Audio Methods: Testing Phase

2/5/04

Audio Test on 2/5/04 Sony Digital Recorder and Lapel Microphone. I recorded myself working with some of the Vhs teachers. The microphone is omnidirectional, I was picking up a lot of background conversations. A normal teacher voice was picked up at 6 foot clearly as if they were right there. The VHS teacher I was working with, has a soft voice. His voice did come through clearly, too, at a range of 2 feet. My voice is clear.

Being that I was there it’s easy for me to put this recording data in context. Without such context, this data might be useless. This is why the video and this audio track might yield interesting data.

I could see it is difficult to match dialogues even with video there is a lot of background noise that is picked up. I should have tried different levels of microphone sensitivity. Especially with a teacher like Betsy, clarity is going to be important. A test with teachers in situ is necessary.

Downloading audio to wav files. The software is easy to use. Just click on the file and save as a wav file, 16bit monoaural. Stereo might be better, but not worth the extra file space. An audio file of 19 minutes and 20 seconds saved as a wav file of 25 MB. It took about 3 minutes to complete the save.

Extrapolations: Hour’s worth=75MB and 10 minutes to save. Each data collection session includes 2 to 3 hours of audio. That’s 150 to 225 MB plus 20 to 30 of downloading that info. For six teachers that 900MB to 1.35GB and 1.8hr to 3 hrs. For six weeks once per week, that’s 5.4GB to 8.1 GB just for audio. That’s a factor of 72.

For video that’s 3.6GB per hour times 72. For these sessions that 260GB! That’s just raw data files. Exported files might take up another 10GB? This part I haven’t tested. We’re looking at a minimum data storage of 300GB. (I was way wrong, 840GB for 60hrs of video)

Logistically, I can’t see being able to do one video session per teacher per week. We don’t have the (wo)manpower. Once every two weeks for two months, seems to me a more realistic timeline. This gives us some logistical flexibility. Lin and Xiao will definitely need to conduct at least two sessions per week throughout the course of the eight weeks. I plan on doing once per week, and doing all of the data storage and manipulation.

I took the time to try out very highly rated wav to mp3 converter. It took another 5 minutes of computer processing time and the mp3 file size is still 18MB for 20 minutes of voice. Definitely not worth it converting to mp3.

Video Analysis Test 1

Recording of Immersion class on 1/28/04, written 2/2/04
Using Hi8 Analog Camcorder

Drawbacks to this technology is that it is not in digital, but can be by using a Digital8 camcorder. Although I tried playing this tape in my Digital8 and it did not play. Perhaps that was a function of my previous camcorder not this current version. Why is digital necessary? It decreases download time, I think. The quality is better. I noticed in playing back this tape, the video was quite pixelated. It was not a sharp crisp picture. Any transfer is probably going to increase loss of fidelity. Digital camcorders are the only way to go.

Lighting is important. Having only a background light, hid the face of the instructor. While we can't bring in extra lights like a movie set, we can make sure that the lighting is appropriate for video recording. This means having light shine on the face of the teacher. And having as little background lighting as possible, this includes light emitted from computer screens.

The audio quality in the straight ahead angle was quite good. However, all of the students were very quiet. There was no background noise. It's hard to extrapolate the sound recording of the camcorder in more difficult conditions. I could see how the camcorder's audio could be deemed useless in a hands-on activity. The classroom is also quite a bit larger than an elementary classroom. There is less echo and reverb than we might encounter in the classrooms. Possible alternative is to use an external unidirectional microphone that picks up better quality sound. It would need to be turned in the direction of conversation.

Coverage is also a problem. I set up the camera on a tripod. I did not man the camera through the class. I was lucky to have a teacher who did not move off screen even though I did not have the whole classroom in the viewscreen. A mobile teacher would demand a high corner angle to get full coverage with an unmanned camera, or someone who turned the camera to follow the teacher. I found that the disturbance of the camera minimal on the tripod, whereas once I started moving the camera around, people did notice that they were now in view. Manning of the camera is good because there can be good usage of the zoom. This allows the video and visuals to speak for themselves much better. However, this will not be feasible for us, especially if we want to take a separate set of field notes. A second person to man the camera still affects the children and maybe the teacher. This causes descriptive validity problems. It's probably a bit unethical to be so obtrusive. Perhaps this is a mode we could turn to in the case of some kind of microanalysis of student-teacher interactions later in the research design. By getting total coverage, we can use the recording of teacher reflection as a separate audio track. This would involve the teacher watching, commenting, reflecting on what happened in the video. This could be done the same day after school or maybe a few days later allowing the teacher some intermediate personal reflection time. Professional development literature suggests that teacher reflection is a powerful way of changing teacher beliefs and improving teaching. This whole process is then an ethical mode of inquiry.

Process for Videotaping and Video Reflection

3/24/04

1. Use the Shot Log Template, recording the time on your watch, record times at changes in classroom behavior (e.g. giving directions then students working in groups).
2. In addition to normal field notes, make note of important incidents you think you might want the teacher to reflect on. This can also be done afterwards, by reading through the field notes.
3. Choose one long 15 to 20 minute section of video from your notes. Correlate those actions to the time.
4. Then cue the tape to that part of the videotape. In the future, these video clips of importance will be the sections we decide to download to the hard drive.
5. Record digitally the audio (using the lapel microphone) from the reflection. Use prompts like “What do you notice?” or “Please describe in your own words what you think is happening here?” These audio files will be overlaid on the video track on the computer to generate a video reflection.

Meeting notes: Shawn Miller, RE: Video Production

4/8/04

He recommended I speak with Charlotte at Mason Media Lab about maybe using their video production equipment, like Avid and Final Cut Pro. (Haven't done it yet)

About captioning, he suggested I speak with Kristine Neuber. She should have or know about software to put captioning into movies. Final Cut doesn’t seem to have that option. He recommended that I use a white or yellow text with black background for easy reading.

The work flow process should be the following
1. Field Notes, identify key video clips
2. Download video clips to Scratch, with timecode burned in (I didn't do this because I use QT and its timecode player)
3. Produce a video with separate audio track or whatever else I want to create. (Could be done in iMovie)
4. Export product to a number of formats, FullDV Quicktime for use in iDVD or Mpg2 for use in Transana. (Use Media Cleaner Pro)

He recommended that I stick with the G4 we already have. I’ll try to buy more memory for it, boost up to 1GB. (Done)

We looked up blank DVD 50 pack price to be $50. (Otherworldcomputing.com) (Done)

Recommended that I check the video playback on a TV. (Not Done)

I also need to make sure I’m burning the timecode into the video clips during download. I said that the Digital8 format doesn’t seem to do this, but I’ll experiment to find out. Final Cut cannot burn this in after the fact. (Not so worried about this anymore)

Memo on Video Data Collection

4/12/04

Video editing is about identifying what you got versus what you want. If they don't match, then you have to either recapture what you need to span the gap or change what you want your final version to look like.

Memo on the Effect of Case Writing

4/27/04, Transformative professional development

Ellinger, E. (2004). Studying students: The effect of case development on teaching. NARST conference.

Qualitative research presenting data on the changes in teaching practice having gone through the process of writing teaching dilemma cases. Five themes emerged out of the interviews of four elementary science teachers: Listening with purpose, Increased desire to understand student misunderstandings, Ongoing reflection on teacher decisions, Collaboration with colleagues fostered, and Designing assessments that provide real data. The author concludes that case writing is a profound method of teacher professional development.

Use of the instructional design process? Reification in technologically appropriate methods. Teachers must understand what technologies are available to them, what each is good for, how to go about designing for each. What about using the CoPLS model of instructional material for teacher professional development. Spanning the offerings of GMU across literacy, science education methods.

Memo on Observations with Video

3/23/04

Let’s see after talking with Ari last night, the sound of Joe’s voice screaming “Don’t leave analysis to the end!” keeps me thinking. It forces me to be observant during observation. I’m coding in a way on the fly. The video taping helps in that I don’t have to focus on what is said, rather I can get in the heads and analyze on the fly. I listen deeply to teacher questions, student answers. I analyze what the teacher says in response (Flanders). I feel like the color analyst on a sports broadcast. Analyzing blow by blow.

Teachers and Bounded Decision Making Memo

12/09/03

Everyone should understand that information is limited and there are costs to collect, process, analyze and refine data. Bounded decisions are optimally imperfect. That the search for more information is a random process generally guided by some rule of thumb. Two people seeking the same information may end up deciding very differently based on the paths they take. They make the best use of information at hand. (Tisdell’s Bounded Rationality and Economic Evolution)

6/29/04

Teaching/Design/Engineering/Economic and Social Considerations/Decision-Making/Pedagogy

Inquiry Design Support System Ideas

2/24/04

If 75% of teaching is design. Do teachers design for LD and ED students? ESL students?

Delivery of just-in-time Professional Development through an EPSS.

Also create a collaborative network of teacher support.

Project must be database driven?

Access to Resources, Knowledge about Resources, Request of Help like Listservs and Discussion Boards.

Memo on Linguistics

Having reread this memo and the Steadman chapter on Classroom Interaction research, I find that there is a natural links between process-product and linguistic research approaches in the classroom.

I wrote this 4/19/04 after a discussion with Ari, fellow doc student:

Initial Question: How is technology used in studying linguistics?

For second language acquisition. How do people use morphenes, like -ing or -ed. The theory is that there is a natural order of gaining learning attributes.

For conversation analysis. Schegloff in California. Look at the detail transcripts of turntaking. Could be used in guided reflection fidelity studies? Markee has done work to use conversation analysis on second language acquistion.

For corpus linguistics. Concordance studies look at samples of language, eg academic languages of oral, written, textbook, test. Francis Butler. Look up the frequencies of concepts, eg cell wall. Each instance plus and minus five words are returned as a search result. Show you how it was used. Of course there are differences, it allows you to design an intervention.

For second language acquisition of adult, intermediate to advanced proficient, learners. The uses of elaboration, adding of text, and simplification, cutting away of text, have positive learning effects.

Ari plans to examine the washback language from the direct instruction, as a measurement of learning? language acquisition? Looking for differences between second language and first language learners.

Concordance software makers: Barlow at Rice U.

References: S. Hunston (2003). D. Biber Corpus Linguistic

Statistical Test of loglinear measures on a continuum of frequencies of 3 or more variables.

Inquiry Meeting 6/23

Next Meeting Wed 8/11, 10am?

Began with the poor Reliability results of the Flanders Interaction Analysis. Xiao and I will continue to refine of coding interpretations. Brenda will begin when she gets back.

We spent most of the meeting sorting out books and readings for next semester. We balanced between Content (Science, Reading, Inquiry) and Process (Instructional Design).

[add list here]

Flanders vs. TIMSS

These are two very different ways of analyzing video of science teaching.

Flanders is more fine-grained. My hope with Flanders is to have the data speak for itself as much as possible. If Inquiry is indeed the intention, then it should "show" up as a pattern.

TIMSS seeks to look for everything under the sun, that we know about inquiry science. TIMSS's look-to-see-if-it-there approach is very interpretive.

Rip Tape Format

The essence of this memo was to discuss questions and implications of the AVI format that is created when a tape is imported into Windows or Mac video editing programs. I wound up using the Mac iMovie (easier to use than Adobe or FCP) to rip tapes and bought Canopus ProCoder to convert formats (cheaper than Cleaner):

I have Adobe Premiere 6.5 DV AVI (type 1 or 2) I don't know. Also will be using Studio 8 to rip my tapes. to digital format. Can I use these in FCP? To generate and edit my movies?

Importing Adobe Premiere files into FCP went fine. So in terms of cross platform, I'm not too worried. I can use an Windows machine to rip my tapes. I can rip and archive to DVD. Then use FCP and Cleaner to compress the movies to MPG, QT, etc.

On Doing Video Analysis

6/13/04

Utter failure. I wanted to watch on my laptop in the library to do Flanders Interaction Analysis. However, I didn't bring headphones, and the video was too large to play on my old laptop (not enough processing power). I might have been better off doing a conversion to MPG1. The playback on Windows Media Player is usually better for this computer than Quicktime. Which I probably would need to do anyway to work with Transana (Although Mac version ready by Aug?) The other disk was a DVD which I don't have a player on this laptop. Gotta go home and do work on the Machine (desktop). Perhaps if I had an Internet connection, I could have watched it on a web stream?

To recap:
No headphones
Laptop too old
QT file too big
No DVD on laptop

InterCoder Reliability

As far as I can tell the three common methods of calculating InterCoder Reliability are the following:

Cohen's kappa
Scott's pi
Krippendorf's alpha

I find that Cohen and Scott's formulas almost the same, they will always yield similar results. According to Krippendorf (1980) they do say different things. I still have to read Chapter 12, I'll blog it when I'm done.

Flanders - Differentiating among categories

Here are the Points of Contention between Xiao and I so far:

2 vs. 3: Simple Okay's, Umhm's, repeating of the answer don't count as 2's. There must be some indication of judgement by the teacher that the answer is a good one.

4 vs. 5: Is reading a question off a worksheet, a question? No. There is no verbal response by the student. I'm still not sure about this, but it feels like this is the right way to go.

8 vs. 9: Xiao and I disagreed over a response to an open-ended question was an 8 or a 9. I said 9 because the teacher is not looking for an answer they have in their own head, but is listening to the student's idea. Also this distinction will help us identify patterns of open vs. closed questions.

Coding across time intervals: I had advised Xiao to code the extra segments, whereas I myself was not doing so. If a code goes into an interval by only a syllable then I don't code it. Whereas if there is a change or sequence of codes, then I do code it. We also resolved to avoid multiple codes in single intervals as much as possible, breaking Amidon's Rule #4. This was in an effort to increase reliability, but might threaten validity of our results.

We also conflicted on the when the time interval started and the code represents the interval before or after the stated time. This was why we were off by 3 seconds overall. We resolved to have the code stand for the stated time until the next stated time. (eg 0:00:03, 5 means the interval 0:00:03 to 0:00:06 was coded 5)

Our overall reliability was 41% for BA_2004_03_29. This is unsatisfactory. Having had the conversation over the above issues, it is hoped that reliability will increase significantly.

We've found that in sections of high interaction, it had been difficult to get reliable results so far. The intervals might need to be reduced to 1 sec in such cases. Xiao and I achieved 58%.

Questioning

Gribble, J., Briggs, S., Black, P., Abell, S.K.

This book chapter raises many interesting points. Questioning turns out to be a game that students either learn to play or fail. There can be different questioning paths, that elicit student thinking and knowledge or maintain control of the classroom. A Why questioning path is focused on a declarative knowledge view of science, whereas a What If? path is focused on an inquiry based view of science. Question need not only probe for understanding but also initiate sensory observation and investigation.

Identify different questioning paths that work in different science teaching and learning situations. Identify What If Patterns of discourse to indentify inquiry?

Chapter Review: Rex, Steadman & Graciano

Researching the Complexity of Classroom Interaction by Rex, L. Steadman, S., & Graciano, M.

This is an extensive qualitative meta-analysis of research in classroom interaction. They identified 7 categories:

1. Process-product
2. Cognitive
3. Situated cognition
4. Ethnographic
5. Sociolinguistic and discourse analysis
6. Critical
7. Teacher research

The authors seem to fall on the side of mixing these categories appropriately to answer particular questions. In essence they are looking at the same thing with different assumptions and agendas.