Trying to get the work life back on track, which includes trying to blog here more regularly.
As some of you may know, I’ve spent the last seven years or so looking closely at programmatic data in technical and professional communication (TPC) and talking to faculty and administrators. I’m diligently (and somewhat desperately) trying to wrap the project up and get all the data and analysis and you know, academic things I want to say all done and out there for your consumption.
Over the course of the next few weeks, however, I’ll be posting different snips and pieces of data. The point of putting some of this data out there is that more people need to have access to it and it doesn’t fit neatly into the book I’ve finishing up. One of the great things about gathering data is that you end up with more than you need. Part of its messiness is actually its beauty.
Other than this announcement as a way to inform but also a public accountability is to lead into a post about research practices. One of the things I’ve been doing in some of my research related work is trying to separate and unpack methodology, practice, and methods. While I appreciate the work of Sullivan and Porter in Opening Spaces, the way their work has been picked up in TPC (and in areas in writing studies) does a dis-service to the research process. Why? Because when they used methodology to include disciplinary ideologies, methods, and practices, it seems scholars lost sight of the fact that these parts are indeed separate and need different kinds of attention.
For me, I have grown more and more discontented with the way we talk about our methods, methodologies, and practices. This is one reason that in RHM we are emphasizing the “methods” section and encouraging authors to make some of their thinking and moves more specific.
Researchers who have not done a lot of messy data gathering are often surprised to learn that the practice of research can change. What I am calling the practice or practices of research are the actual day to day, moment to moment decisions made during research. One practice of research that was a problem for me from the very beginning of the programmatic project was what to collect. Some might claim this to be part of the methodology or the ideological assumptions behind the questions, but for me it was a practice question. My method was textual analysis (though I chafe at that and will explain about that in a different post), but part the research study design was determining what to gather, which was part of the practice.
As is the case with a lot of research projects, I started with a specific goal in mind, but I was continually side tracked because of other things that I found that were interesting. The first programmatic piece I did was on Masters programs in TPC. I initially was trying to follow previous programmatic research and knew early on that there method and even more so, methodology was not suitable for what I wanted to do. But when I made that decision, I was weeks into the project when I realized the ramifications of that decision. And what were those ramifications? Well, I realized that what I wanted to do was extremely labor intensive and tedious.
Some of the early practice decisions I made and later regretted was I started by simply looking at websites. I was about 30 or 40 schools in when I realized the mistake because I had just had a conversation with a program administrator who was annoyed that they couldn’t get their website content to be updated to the actual requirements. That was when I decided I had to use course catalogs. Thus, I had to circle back and re-confirm every data point. I also didn’t gather data in a systematic way, which early on the process meant I had a lot of holes that eventually had to be filled. The biggest thing was that I initially didn’t really gather course information. I just had gathered the requirements, but when I went to start analysis, I realized that without the course information, I would have trouble making any sort of generalizable claims. Thus, I had to circle back around again.
This is just a brief snapshot of some of the problems with my practice during the research process. During the ensuing years, I have honed and made more specific what my practice was and why I did that practice. Part of that was just experience from working in and trying to manage a program and part of it was experience from the research process itself. For a project like this one, I’m not certain some of the practice problems could have been avoided in the early stages simply because of my inexperience with programs and inexperience with this specific type of research/data gathering.
There were some very practical practice decisions made along the way, too. I had to abandon the inclusion of two-year colleges and international programs (though I have data on both). Each type of degree program has its own set of definitions and limitations on exclusion criteria and that couldn’t be used across degree programs. Making peace with this messiness was not always easy, and it caused the thinking and re-thinking on how to align criteria or not.
I then made the decision early on to include a form of inter-rater reliability to the data set. I recognized that I had, I have, a unique view of what I think I program should be and that view may not be universal. It’s also true that you can ask faculty in the same program to characterize their program (which I have done) and they will answer differently. So all the data has been examined by other academics, as well as practicing technical and professional writers/communicators. This step added much time and coordination, but it helps provide a validity and trustworthy factor to the data that most programmatic data sets don’t have.
The bigger issues around practice, and this is another case where practice pushes up against methodology, is when Sally Henschel and I were working on our 2013 piece on 
undergraduate programs. We were using a combination of my own research on masters programs and Sandi Harner’s work from 2005. We had to determine how to code the courses in a way that would make sense to other people not embedded in thousands of course titles and course descriptions and a way that would produce codes that could be analyzed. That is, we couldn’t have too many and we couldn’t have too few. The practice of coding a small number to see what patterns developed, honing that system, analyzing again, and then moving to a larger number worked fairly well. We were able to devise a system that accounted for the wide variety of courses, but the same system could still yield data that could be discussed in generalizable terms. What started as something like twenty something codes (I’m trying to trace this process to get more specificity to it) ended up being 40 primary codes and handful of secondary codes. Working with these codes over the last two years as I’ve been updating and expanding the original published work has shown that the decisions that Sally and I made were not only good decision, but they were sustainable decisions. *
Another practice question is about the timeliness of the data. I started the book on this data about 4 years ago. Right after the undergraduate majors piece came out. But I got frustrated as I was writing at that time because some of the data had aged a bit. And when I went to my network of advisors and mentors, there was absolutely no consensus on the best way to proceed. Some said use the clean data I already had and fill in the holes for what I wanted to add. Other said that all the data needed to be current. Others gave varying answers in between these two while also providing (unsolicited) advice on what additional data I needed to include or what data I should leave out.
Did I mention that this type of work is labor intensive?? Let me tell you how labor intensive. I am so blessed to for the first time have a research assistant to help with this project. She has been updating the minors data information. There are 168 4-yr institutions that offer a TPC minor in the US. She has put in 60 dedicated hours and is not finished yet. And that’s just the time to gather the data. That’s not the coding of it nor the analysis of it. **
This little sidebar on the time it takes circles back to the practice question I had. Every time over the last three years I started to work on this project again, I got bogged down in the question of the data. This summer (before my life fell apart) I had finally made decisions on how to proceed. But this part of talking about the practice of research should show how something as “simple” as gathering data can actually be much more complicated.
Speaking of complicated: all of these practice decisions definitely get more complicated (and even nuanced) when working with people and communities. I have a piece in a review that lays out some of the practice concerns with working with communities based on years of working with and in different communities around health and environmental health issues. If you remember nothing else from this post it’s that we–researchers and teachers–need to be thinking more carefully and talking more about the research practices that underscore our work.
So what decision did I make about the programmatic data? Well, it’s complicated and I’m trying to write my way into the description of it in part by this post. In short, however, the vast majority of the data will be current to the 2016-2017 catalogs. By the time the whole book is done and reviewed the data will still be less than a year out, which is probably the best anyone can do to make data current for a project of this scope and with the fluidity of academic programs. No matter, it will provide a sealed moment—a snapshot—of the curricula definition of what TPC was/is at that time.
*This is why separating out practice from ideologies is so important. It gives you space to figure out some of the details that may have been overlooked had the research moved forward with an overall methodology (using Sullivan and Porter’s term).
** Now consider that the larger project will report on over 600-degree programs as well as information on outcomes, competencies, and maybe some job market data and you can see how the time commitment involved.