The Practicality of SoTL

I love the serendipity of reading, reviewing, thinking, etc. For a while I’ve been thinking about a different type of scholarship that I feel I’ve been doing. It is something that emerges out of my context as a researcher at the community college. Although I am invested in doing serious critical research, I am in an environment that focuses on the practice of teaching. Research is both above and beyond my direct responsibilities, teaching, and is really only appreciated if it is directly related to and useful towards my teaching.

As I was reading an article to check a reference in the edited collection I’m working on, Rhetorically Rethinking Usability I came across a table (figure 1) that compares characteristics between usability testing and traditional experiments (Dieli, 1989).

Figure 1: Some Characteristics of a Usability Test Compared with Those of a Classic, Controlled, Experiment

Characteristic	Classic, Controlled Experiment	Usability Test
Goal	Scientific discovery	Help produce an adequately useful product
Problem	Research leads to problem definition	Problem defined by client
“n”	Large; can run statistics as a data analysis tool	Small; just enough to see patterns
Confounds	None (theoretically)	Some we can live with
Time	Can take a long time	Must take a short time
Pilot	Run to ensure validity and reliability	“Dress rehearsal” to ensure client buy-in
Report	Traditional format	Short, plus meeting with client

(Dieli, 1989, p. 275).

In the article Dieli (1989) was discussing how the context of actual usability testing requires flexibility and adaptation in a manner that can not replicate nor produce the methods and products desired by traditional researchers. I think people doing Scholarship of Teaching and Learning (SoTL), especially those of use at a two-year college, can identify greatly with Dieli’s point. I’ll admit, I’m tired of trying to explain the how and why it is almost impossible for me to construct a methodologically “sound” empirical study in my SoTL research about teaching with technology. I think the table above provides some ways to think about how and why my frustrations are valid.

Figure 2: Some Characteristics of the Scholarship of Teaching and Learning with Those of a Classic, Controlled, Experiment

Characteristic	Classic, Controlled Experiment	SoTL
Goal	Scientific discovery	Help produce an methods to better facilitate teaching and learning
Problem	Research leads to problem definition	Solution tested by researcher
“n”	Large; can run statistics as a data analysis tool	Small; just enough to see patterns
Confounds	None (theoretically)	Some we have to live with
Time	Can take a long time	Unless conducting over numerous terms, only as long as a class cycle.
Pilot	Run to ensure validity and reliability	Unless conducting over numerous terms, revising the teaching method, and therefore the research method, each term for a new class.
Report	Traditional format	To be timely it should be reported in more quickly published and easily consumed format/media; however, it can also follow in traditional format.

By filling in the second table, I’m better able to articulate how/where there is breakdown between the work I want to do and produce with more “traditional” concepts of scholarship and publication. The “n” row acknowledges that these types of studies will rarely ever be “statistically significant.” This implies two things: first, more qualitative methods need to be included and accepted as valid; and second, that “experience” and “lore” do need to be given some credit an authority.

The “confounds” row acknowledges that classrooms are not sterile environments, period! Again, there will always be considerations about other contributing factors. The “time” row shows were we begin to see the possibilities to different types of SoTL research methods and publications emerging. Obviously an teacher/research can develop a traditional empirical study that uses some classes as the experimental group, others as control, and repeat it over different semesters to get a statistically significant number of participants. Reality check in my world, why would I teach two sections of the same class using different methods, especially if I’m teaching four or five courses? And then, ethically, if I sincerely believe a method is going to better facilitate learning, am I going to not do it with other students (I guess the medical field has to deal with this ethical dilemma all the time, I’ll have to ask my dad about it).

As I started to explain in the discussion of the “time” row, you could either be doing SoTL research that is much more “traditional” and use a whole session’s worth of classes as pilots. However, if you do SoTL research more like I do, either every session is a “pilot,” or they are never “pilots.” Since I like to research how new technologies better facilitate teaching and learning, I’m always trying a new technology or a newer technology in a different manner.

This always beta method of research is what I’m thinking of when I talk about Just in Time Scholarship (JiTS). Basically the research is always in pilot or process; therefore, the “reporting” or publication is just a snapshot in time of the results. Obviously this can be published in more traditional modes, journals and books; however, with the slow time to publication of those media, the content is woefully outdated by the time it is actually read by the intended audience. The JiTS is much better suited for publication in blogs and wikis. Both the publication and critique/commentary processes are faster. And just because its blogging and wikis doesn’t mean it can’t still be peer reviewed! Or, if the person is publishing “results” regularly, think blogging, the peer review can occur after the “publication” of the information in the reply section of the blog posting.

References
Dieli, Mary. (1989). The usability process: Working with iterative design principles. IEEE Transactions on Professional Communication 32(4): 272-278. Retrieved July 5, 2007, from IEEE Electric Library Online.

July 07 2007 | Posted in 21st Century Scholarship