Forums: Community: The Soap Box:
Grade level of congressional speeches and ideology
RSS FeedRSS Feeds for The Soap Box

Premier Sponsor:

 


jt512


May 24, 2012, 1:15 PM
Post #1 of 19 (1599 views)
Shortcut

Registered: Apr 11, 2001
Posts: 21887

Grade level of congressional speeches and ideology
Report this Post
Can't Post

http://blog.revolutionanalytics.com/...analyzed-with-r.html


lena_chita
Moderator

May 24, 2012, 4:35 PM
Post #2 of 19 (1578 views)
Shortcut

Registered: Jun 27, 2006
Posts: 5466

Re: [jt512] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post


I have a sneaking suspicion that the majority of Congress members won't be able to interpret those plots, either... Unsure


curt


May 24, 2012, 6:20 PM
Post #3 of 19 (1570 views)
Shortcut

Registered: Aug 26, 2002
Posts: 18226

Re: [jt512] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

In reply to:
While the speech levels of members of both parties have declined over the past couple of years, the chart below shows that for Republicans, the more ideological extreme the congressperson, the less sophisticated the speech.

Well, there's a shock. Cool

Curt


Partner rrrADAM


May 25, 2012, 3:20 AM
Post #4 of 19 (1544 views)
Shortcut

Registered: Dec 19, 1999
Posts: 17543

Re: [curt] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

You think the Teatards trended the curve down?


chadnsc


May 25, 2012, 6:15 AM
Post #5 of 19 (1528 views)
Shortcut

Registered: Nov 24, 2003
Posts: 4449

Re: [rrrADAM] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

I read a related article that stated the reason why congress on average have an 8th grade vocabulary in speeches is because they are 'speaking to the people' and need to 'speak in terms that the country can understand'. Unsure


hyhuu


May 25, 2012, 7:40 AM
Post #6 of 19 (1518 views)
Shortcut

Registered: Jul 25, 2001
Posts: 491

Re: [chadnsc] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

chadnsc wrote:
I read a related article that stated the reason why congress on average have an 8th grade vocabulary in speeches is because they are 'speaking to the people' and need to 'speak in terms that the country can understand'. Unsure

Damm!! I need to work up to the 8th grade level because most of their speeches sound like garbages to me.


chadnsc


May 25, 2012, 7:52 AM
Post #7 of 19 (1511 views)
Shortcut

Registered: Nov 24, 2003
Posts: 4449

Re: [hyhuu] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

hyhuu wrote:
chadnsc wrote:
I read a related article that stated the reason why congress on average have an 8th grade vocabulary in speeches is because they are 'speaking to the people' and need to 'speak in terms that the country can understand'. Unsure

Damm!! I need to work up to the 8th grade level because most of their speeches sound like garbages to me.

Hooked on Phonics worked for me.


styndall


May 25, 2012, 8:48 AM
Post #8 of 19 (1503 views)
Shortcut

Registered: May 29, 2002
Posts: 2741

Re: [jt512] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post


As much fun as it is to call tea partiers dumb, this grade level metric is one of the dumbest things speech science has ever produced. It's just a function of words per sentence and syllables per word. Trends in speeches come and go, and it shouldn't be surprising to see changes in this measure over time, totally unrelated to actual education or clarity of ideas. There are interesting metrics of idea and speech complexity out there, but Flesch-Kincaid isn't one of them.

It'd be really nice to see someone take this corpus of speeches and apply UGA AI Institute's CPIDR, a process that use some parsing and a semantic proposition engine to find a ratio of novel ideas or propositions per word in texts. I certainly don't have the time right now, though.


Partner cracklover


May 25, 2012, 11:23 AM
Post #9 of 19 (1485 views)
Shortcut

Registered: Nov 14, 2002
Posts: 9935

Re: [jt512] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post


Assuming that the methodology is any good (which, given the post right before mine, might be a bad assumption) it's actually quite interesting.

One thing that pops out right away is how high the centrist republicans score. This is, when you think about it, completely intuitive. These are the people willing to stand up and fight against an incredibly nasty back-biting party, and for legislation that might be controversial, but presumably has merit.

I think the biggest harm being done to our political system now is not the driving out of excellent liberals like Barney Frank, but excellent moderate republicans like Olympia Snowe.

GO


jt512


May 25, 2012, 7:13 PM
Post #10 of 19 (1458 views)
Shortcut

Registered: Apr 11, 2001
Posts: 21887

Re: [cracklover] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

cracklover wrote:

Assuming that the methodology is any good (which, given the post right before mine, might be a bad assumption) it's actually quite interesting.

One thing that pops out right away is how high the centrist republicans score. This is, when you think about it, completely intuitive. These are the people willing to stand up and fight against an incredibly nasty back-biting party, and for legislation that might be controversial, but presumably has merit.

I think the biggest harm being done to our political system now is not the driving out of excellent liberals like Barney Frank, but excellent moderate republicans like Olympia Snowe.

GO

The graph I linked to is an analysis of data collected yearly since 1996. Therefore, Styndall's criticism, that the apparent grade level of speech measured by the scale used would vary over time, potentially invalidates the analysis, because, as far as I can tell, time was not controlled for. Plausibly, the effect of time could explain the entire negative relationship between grade level and conservatism among Republicans. This would be the case if the grade level of speech declined due to a secular trend, and the more recently a Republican has been newly elected to Congress, the more conservative he or she is (which is probably the case).

The authors of the study also performed an analysis using data from a single session of Congress, which obviously cannot be confounded by a secular trend. The results of this analysis were similar to their multi-year analysis. However, the authors' explanation of their regression model is unclear, and I don't trust it. Fortunately, the authors provided their dataset on their website, so I downloaded the dataset and did my own analysis of the single-session data. My results are qualitatively similar to theirs. I found no significant relationship between ideology and grade level of speech among Democrats, but a highly significant relationship among Republicans, with greater conservatism predicting lower grade level of speech.

Although these relationships cannot be confounded by a secular trend in apparent grade level of speech, they could be confounded by time in office or the age of the member of Congress. The dataset included the former variable, so I was able to control for it, but not the latter, which remains a potential confounder. I also controlled for the geographic location of Congressional district or state, albeit rather crudely. There are undoubtedly other potential confounding variables, which, perhaps, styndall could shed some light on.




(This post was edited by jt512 on May 25, 2012, 11:56 PM)


styndall


May 25, 2012, 7:50 PM
Post #11 of 19 (1453 views)
Shortcut

Registered: May 29, 2002
Posts: 2741

Re: [jt512] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

It's not confounds that you need to be looking for, really, but rather, the trouble is with the idea of 'grade level.' Consistent short word and sentence length can be indicative of poor control of language, but it can also be a stylistic choice. In fact, style manuals often suggest using shorter words and phrases in place of longer ones whenever possible - see Strunk and White, Orwell's Politics and the English Language, etc..

Attaching a term like 'grade level' to a such a measure just serves to attach a negative value judgment where none may be warranted. You may as well take the grade level measurement and call it 'degree of word inflation' or 'vocabularic devaluation index' or some such. Absent other information about the actual content of the speech in question, it's meaningless.

Like I said above, there are tools out there for doing things like measuring logical propositions per word in a given text, but none are in use here.

EDIT: By way of example, here's the bit of Ecclesiastes and it's terrible-style translation from Orwell's Politics and the English Language.

The original:

"I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all."

Orwell's Parody of Terrible Writing:

"Objective considerations of contemporary phenomena compel the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account. "

The first gets a grade level score of 20, via a calculator I found just by googling. The second gets a grade level of 32. Thus it's easy to see that the grade level score doesn't necessarily mean anything about quality of speech or ideas.


(This post was edited by styndall on May 25, 2012, 8:00 PM)


jt512


May 25, 2012, 8:11 PM
Post #12 of 19 (1446 views)
Shortcut

Registered: Apr 11, 2001
Posts: 21887

Re: [styndall] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

styndall wrote:
It's not confounds that you need to be looking for, really, but rather, the trouble is with the idea of 'grade level.' Consistent short word and sentence length can be indicative of poor control of language, but it can also be a stylistic choice. In fact, style manuals often suggest using shorter words and phrases in place of longer ones whenever possible - see Strunk and White, Orwell's Politics and the English Language, etc..

Attaching a term like 'grade level' to a such a measure just serves to attach a negative value judgment where none may be warranted. You may as well take the grade level measurement and call it 'degree of word inflation' or 'vocabularic devaluation index' or some such. Absent other information about the actual content of the speech in question, it's meaningless.

Like I said above, there are tools out there for doing things like measuring logical propositions per word in a given text, but none are in use here.

So, basically, the more conservative a Republican (but not a Democrat), the more likely he is to use shorter sentences comprised of shorter words. So, why would that be?

Jay


styndall


May 25, 2012, 8:31 PM
Post #13 of 19 (1441 views)
Shortcut

Registered: May 29, 2002
Posts: 2741

Re: [jt512] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

Explanations from poor control of language to intentional use of a plain, folksy style are possible. Word length and sentence length are, among other things, social indexes, and all speakers of language alter their speech to index different characteristics in different situations. Tea partiers seem to have a strong interest in distancing themselves academics, and speaking style can be one of the methods of getting that distance.

That's just an off-the-cuff hypothesis. Like I've been saying, you'd need to look at the actual content of the speeches (along with lots more stuff) to draw conclusions about tea party language use.


atg200


May 26, 2012, 7:48 AM
Post #14 of 19 (1415 views)
Shortcut

Registered: Jul 27, 2001
Posts: 4317

Post deleted by atg200 [In reply to]
Report this Post

 


petsfed


May 27, 2012, 11:26 AM
Post #15 of 19 (1383 views)
Shortcut

Registered: Sep 24, 2002
Posts: 8585

Re: [jt512] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

I wonder how the assignment of their ideology rating works. Its not clear if they assumed a linear distribution from lunatic to centrist and assigned values from there, or if it was some measure of how often the other party was mentioned, or some other metric entirely. The point is, without that information, I'm inclined to say that the stated trends are meaningless.

It could very well be the case that their measure of ideology includes the tendency to reduce things to black/white situations, which is one way to spoof the grade level, at least as it pertains to Microsoft Word's grade level calculation (I would wager that they simply dumped the speeches into Word and used the review feature to get readability statistics).


yanqui


May 28, 2012, 6:15 AM
Post #16 of 19 (1352 views)
Shortcut

Registered: Feb 24, 2004
Posts: 1550

Re: [atg200] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

As hard as they may try, I doubt any of them will ever reach the Zen-like perfection achieved by the Dubya. Maybe he was right: we've misunderestimated him.


(This post was edited by yanqui on May 28, 2012, 6:17 AM)


Partner cracklover


May 29, 2012, 8:04 AM
Post #17 of 19 (1296 views)
Shortcut

Registered: Nov 14, 2002
Posts: 9935

Re: [petsfed] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

petsfed wrote:
I wonder how the assignment of their ideology rating works. Its not clear if they assumed a linear distribution from lunatic to centrist and assigned values from there, or if it was some measure of how often the other party was mentioned, or some other metric entirely. The point is, without that information, I'm inclined to say that the stated trends are meaningless.

You're misunderstanding. Two different unrelated metrics: the ideology is based off voting record, while the grade level is based off speeches.

GO


jt512


May 29, 2012, 10:29 AM
Post #18 of 19 (1278 views)
Shortcut

Registered: Apr 11, 2001
Posts: 21887

Re: [petsfed] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

petsfed wrote:
I wonder how the assignment of their ideology rating works. Its not clear if they assumed a linear distribution from lunatic to centrist and assigned values from there, or if it was some measure of how often the other party was mentioned, or some other metric entirely. The point is, without that information, I'm inclined to say that the stated trends are meaningless.

It could very well be the case that their measure of ideology includes the tendency to reduce things to black/white situations, which is one way to spoof the grade level, at least as it pertains to Microsoft Word's grade level calculation (I would wager that they simply dumped the speeches into Word and used the review feature to get readability statistics).

Their measure of ideology was based on voting records, not the speeches.

Edit: GUed


(This post was edited by jt512 on May 29, 2012, 10:29 AM)


hyhuu


May 30, 2012, 6:37 AM
Post #19 of 19 (1211 views)
Shortcut

Registered: Jul 25, 2001
Posts: 491

Re: [jt512] Grade level of congressional speeches and ideology [In reply to]
Report this Post
Can't Post

So according to this analysis, at what "grade level" is Lincoln's Gettsburg Address? Simple, short and concise.


Forums : Community : The Soap Box

 


Search for (options)

Log In:

Username:
Password: Remember me:

Go Register
Go Lost Password?
$16.16 (10% off)
$14.36 (10% off)
$14.36 (10% off)
$21.56 (10% off)



Follow us on Twiter Become a Fan on Facebook