Undergraduate education and the aura of expertise



Will writes from Washington, D.C. (well, Arlington, Virginia). You can reach him at willblogcorrespondence at gmail dot com.

Related Post Roulette

34 Responses

  1. Avatar the innominate one says:

    The accreditation agencies are asking for colleges to perform assessments to determine whether or not students are actually acquiring the skills their classes are supposed to be conveying. Much scrambling, plotting, planning, and wailing and gnashing of teeth is occurring as university departments attempt to measure the efficacy of their programs.Report

  2. Avatar D.A. Ridgely says:

    And here I thought GWU was really a real estate investment trust disguised as a university for tax purposes!Report

  3. Avatar Rufus F. says:

    Appropriately enough, I’m reading this while on campus, so I’ve got to be brief for now. But I should note that I’m really a rogue classicist, since I’m studying the classical era for fun and profit, while nominally a historian of the Early Modern period- actually, I’m much more of a generalist than people who are more successful in academia!

    Secondly, I should note that I’ve sent fellow academics, particulalrly those who are tenure-track into fits of sputtering rage (almost) by calling for an exit exam for undergrads. It’s quite a controversial idea, which suggests to me that the demand is growing. What academics say if you use dirty words like “assessment” is always, “But you can’t quantify what we teach!” Sure. I know that. Nevertheless, I can usually tell when I encounter liberal arts graduates who haven’t learned what they were supposed to while in college. I’m guessing the people hiring them can too.

    Finally, to some extent, it probably has always been the case that the student himself is the final determinant of how much a particular student learns. It just didn’t seem the way to us nerds when we were undergrads.Report

  4. Avatar James Hanley says:

    inominate one,

    Regrettably, the assessment that the accreditation agencies are asking for measures only the aura of success, not the real success itself. I have been furiously objecting to the assessment my department has had to do for the North Central Accrediting Association because a) it is clear that all they are really asking for is that we give them a report that they can stick on their shelf, that says we did assessment; and b) nobody has yet been able to explain to me how to do real meaningful assessment of my program.* Not that folks wouldn’t wail and gnash their teeth if they were really having to undergo serious scrutiny, but we shouldn’t confuse the current wailing and teeth-gnashing as a reliable signal that actual scrutiny is occurring. Personally I can hardly even work up the energy to make a good fake product, since I know nobody’s actually going to pay any real attention to what’s between the covers.

    As a policy person, I’d be interested in real assessment. I think we’ve made some great changes in our department over the past few years, but all I’ve got to go on is my gut. I’d prefer to either find out we’re on the wrong track and need to change, or have solid data I could wipe people’s noses with.
    * Here’s a real scam: a large number of people are calling themselves assessment experts because they write lots of articles about assessments. 99% of those articles have some version of the following, “there’s no one right way to do assessment, and each program should find out what works for it.” In other words, these self-proclaimed experts really don’t know anything, and will obliquely admit it, but rather than go to the effort to actually do real research and find out what works, they just study what people do without regard for whether what they’re doing amounts to anything more than painting the leaves green.Report

  5. Avatar E.C. Gach says:

    Unfortunately for us college folk it’s a catch 22, with employers complaining that we “don’t have the right skills,” while still requiring a degree from those scheming universities that confer them. What’s a grad to do?

    And maybe I’m wrong, but I think I recall businesses and companies saying the three most important skills were something along the lines of communication, organization, and ability to work in teams. As much as we can bash the colleges for the amorphous knowledge they declare to bestow, I haven’t seen some clear cut list from people on the employer side making it very easy.

    Should I get a college degree that teaches me skills specific to one specialized industry? What happen to Companies training on the job? Give me a job in the mail room and I’ll work my way up. If only though, as even those jobs no longer exist.

    Where have all the “entry level” jobs gone?Report

    • Avatar Jaybird in reply to E.C. Gach says:

      In the 90’s, I remember seeing the occasional request for people with 10+ years experience using Linux. Given that Linux was kinda created in 1991, this didn’t make a whole lot of sense in 1996.

      There’s also the whole “Job Requirements: Electrical Engineer. Job Description: Change lightbulbs, manage extension cords” issue that always shows up in recessions.

      There is a significant part of me that wonders if this isn’t the evolution of discrimination against particular sub-cultures manifesting itself. “You want to demonstrate that your culture and the corporate culture can co-exist? You need a degree.” Once upon a time, a degree wasn’t necessary… but the things that happened under those circumstances were found to be illegal. So they evolved into something legal. That’s it.Report

      • Avatar Trumwill in reply to Jaybird says:

        Several years ago, I was working for a company in an XML programming department of about 20 or so where I was the junior manager of said department. We had some jobs to fill (paying XML programmers under $10/hr will do that) and the new job requirements were so stringent that nobody in the department, including the senior manager, was qualified for an entry-level position. The senior manager didn’t have a college degree. I was six months shy on “industry experience” (as in the industry our IT company served, not IT industry experience).

        The end result was that nobody ever applied and we scambled to find people we knew that didn’t have the requisite experience but that we could convince the right people to hire anyway.Report

      • Avatar E.C. Gach in reply to Jaybird says:

        I’d love for a third party industry to crop up, and a lot of recruiting services do this in some ways, where you can go and get tested as to your ability or skills in certain areas. For instance I might be proficient with graphic arts software, or web programing, even if I don’t have the requisite experience. How nice it would be to go somewhere and take an “exam” for such things, bestowing upon me some amount of legitimacy that would otherwise require a transcript or work reference.Report

  6. Avatar the innominate one says:

    Mr. Hanley:

    I can see where certain forms of assessment might only measure the aura of expertise rather than expertise itself, and I see and agree with your point that ideally the aura of expertise and expertise itself should have a correlation coefficient as near as possible to 1.0. Certainly assessment itself should be assessed as well, until we’re caught in a Mobius loop of meta-assessment ad infinitum.

    That said, it’s a double-edged sword to allow departments to choose their own assessments. Some departments will choose assessments which measure only the aura and not the expertise proper. Hopefully, most will choose useful, meaningful methods which measure expertise gained and eventually, incrementally, cumulatively students will improve. The department in which I’m currently a grad student has a favorable ratio of young, idealistic faculty who are trying to make positive change, at least until they get tenure.

    Realistically, though, shelves will fill up with binders containing TPS reports and assessments and no real change will occur.Report

  7. Avatar James Hanley says:

    Meh, I’d settle for something that was far below a correlation of 1.0 if I could just have confidence that there was some reliable correlation at all. The problem is not just that we’re choosing our own assessments. If we were choosing from some off-the-shelf set, all of which had some known validity, I’d be ok with it. The problem is that we’re all pretty much creating our own from scratch. That takes up my time and pretty much guarantees a crappy product. I think that’s what ultimately has me so tremendously pissed off about the whole thing–I’m being forced to invest large amounts of time in producing crap. It’s like working in a Soviet consumer goods factory, except I’m not even receiving any compensation for it.Report

  8. Avatar E.C. Gach says:

    “if there are no standards for success, there’s no way to actually hold colleges and universities accountable for what they’re teaching.”

    I agree, though the standards should be decided either by the labor market or third parties like recruiter services, not the educational institutions themselves.

    Again, there remains the problem, that Jaybird gives a theory for, about this disconnect from bother sides, that is, both employers and colleges. If employers gave colleges a clear list, of wanting graduates to be able to do A, B, and C, can you think of any reason why colleges wouldn’t seek to accommodate them? Yet I don’t see employers being very clear about what it is the college grads are lacking, even while the labor force shrinks, becomes more productive, and corporations are raking in record profits (Chait at TNR for more on this).Report

  9. Avatar Will H. says:

    I’m thinking that, were internships given a more important function, and especially at the undergrad level, a lot of the problem would go away.Report

    • Avatar Will in reply to Will H. says:

      Ironically enough, I have a few friends who went to GW and their curriculum emphasizes getting internships in DC during the academic year.Report

    • Avatar Rufus F. in reply to Will H. says:

      I’d have to ask my wife, but when she was at Waterloo (Ontario) as an undergrad, she did internships, and I’m pretty sure she said that they (undergrads in Ontario) all do them, no matter the degree.Report

    • Avatar E.C. Gach in reply to Will H. says:

      Everyone should work for free in order to obtain skills, at the very least, rather than pay thousands in tuition, you can literally work for your education. Of course, why not cut out the college all together, and just be trained on the job, full stop.Report

      • Avatar E.C. Gach in reply to E.C. Gach says:

        I agree, which is why I think universities with co-op systems ca not be praised enough, e.g. Drexel and Northeastern.

        Of course those schools are extraordinarily expensive, despite costs being buffered by the intermediate semesters of paid work. I still never understood why anyone would ever do an internship for “course credit” where you not only work for free, but actually end up paying the university for the internship.Report

      • Avatar E.C. Gach in reply to E.C. Gach says:

        The above is directed at you Han.Report

    • Avatar James Hanley in reply to Will H. says:

      The students in my department are required to do internships. That was one of the changes I made.

      Re: E.C. Gach–The point of requiring internships is that we recognize classroom education isn’t the only way to go. Of course that cuts the other way, too, which is why throwing out the baby with the bathwater and doing only on-the-job education may be less than an ideal solution also.

      Truthfully, the most well-rounded, intelligent, and thoughtful people I know almost inevitably combine a liberal arts education with lots of work experience.Report

  10. Avatar D.A. Ridgely says:

    Isn’t what an undergraduate education in general is supposed to accomplish a logically prior question to whether or how one might go about assessing a school’s success?Report

    • Avatar Will in reply to D.A. Ridgely says:

      I suppose so. But isn’t that another problem with a liberal arts curriculum? Beyond a few rote phrases (“provide a framework for critical thinking” etc), I’m not sure what a liberal arts institution is supposed to do.Report

      • Avatar Jaybird in reply to Will says:

        It’s training for you to teach yourself anything.Report

        • Avatar tom van dyke in reply to Jaybird says:

          I needed some cheap credits for my degree, so I took a course in Industrial Management, along with night-class types who actually, you know, worked for a living. Making things and stuff.

          One of the most beneficial courses I ever took, still use what I learned there. Mebbe Liberal Arts should “liberalize” a little.Report

          • Avatar James Hanley in reply to tom van dyke says:

            Actually, I suspect most liberal arts college offer such courses, if not that specific one. See the courses available in my College’s Accounting and Business Administration department, for example.

            I think rather than lack of availability, the problem is that most such colleges, including mine, are in small towns, so we don’t have much of a working-for-a-living-and-taking-night-classes crowd to add there perspective in the classroom for the 18-22 year old students. Short of moving our campus to the city, there’s not much we can do about that. That’s too bad, because the younger crowd benefits from working in classes and project groups with the older and more experienced crowd. That’s one area where a lot of community and regional publics have a real advantage over us. But generally those types of courses are offered.Report

            • I understand, James. All I was saying was that I took Industrial Management quite by accident—it never would have occurred to me as an English major.

              Mebbe there should be a list of electives, a Column B, of stuff a liberal arts major would find completely alien. Real World 101.Report

              • Avatar James Hanley in reply to tom van dyke says:

                I hear you. Requirements differ among schools, as does the role of advising, so the likelihood of students taking those classes differs from place to place and adviser to adviser, I imagine. Listening to a bunch of political consultants last week, I realized that any of my students who are interested in political campaigning should all be taking marketing and management courses–and probably could skip 90% (or more) of their political science classes.Report

        • Avatar James Hanley in reply to Jaybird says:

          It’s training for you to teach yourself anything.

          I fully agree with this. The question is, how do we assess that? Or put another way, how would the theoretical third-party industry that tests for it (that E.C. Gach suggests above) manage to test for it?

          And I don’t mean that as the typical professoriate snark that “what we do can’t be measured by mere quantitative methods.” I believe in quantiative methods, and I want this “learning to learn” skill to be quantitatively testable.

          I think figuring out how to do so is crucial for the future of American education.Report

          • Avatar E.C. Gach in reply to James Hanley says:

            It would be expensive no doubt, but so is every evaluation, SAT, GRE, LSAT, psyche evals, physicals, etc.

            I would propose something along the lines of going in to the evaluation, you are given a certain topic and set of problems related to that topic. You would then have 24 hours to prepare a presentation and memo regarding what you understand of the topic, what factors relate to the problems, how would you would go about solving them, what you would take into account, what solutions others in the field may already have tried, why those will or will not work, etc.

            You would then go in day II for a group evaluation (something to evaluate your capacity for “group intelligence”) where the group would be given a topic and set of problems, that then must be brain stormed, memo-ed, and presented within a simulated 8 hour work day period. The group meetings would be watched by an evaluator, one that was in another room perhaps.

            What do you think of that for a rough basis?Report

            • Avatar James Hanley in reply to E.C. Gach says:

              I can’t necessarily critique it substantively. But it’s simply not practical. I can see a business using it as a process for hiring executive talent, but not as something that could be applied broadly, like an exit exam. And I think we need to try to find something that can be applied more broadly. So it needs to be relatively inexpensive.Report

              • Avatar E.C. Gach in reply to James Hanley says:

                Why wouldn’t it be practical?

                You don’t think something like that could be implemented for $500 a pop? And the above was crafted with regard to Jaybird’s “learning to learn” skill.

                Thousands of dollars are paid per semester, $500 for an exit evaluation doesn’t seem to be a burden.Report

          • Avatar E.C. Gach in reply to James Hanley says:

            I’ll add that these third party evaluators would obviously have an incentive to format their eval criteria to the needs of employers, with all employers listing which eval agencies they accept “scores” from.Report

  11. Avatar Hyena says:

    According to the Cambridge Handbook of Expertise and Expert Performance, there generally is NO transferable “general expertise” to be found.Report