Essay-Grading Software Viewed As Time-Saving Tool
Teachers are looking at essay-grading software to critique student writing, but critics point out serious flaws into the technology
Jeff Pence knows the easiest way for his 7th grade English students to improve their writing is to do more of it. But with 140 students, he would be taken by it at the very least two weeks to grade a batch of these essays.
So that the Canton, Ga., middle school teacher uses an online, automated essay-scoring program that allows students to have feedback on the writing before handing in their work.
“It doesn’t let them know what direction to go, however it points out where issues may exist,” said Mr. Pence, who says the a Pearson WriteToLearn program engages the students almost like a game title.
A week and individualize instruction efficiently with the technology, he has been able to assign an essay. “I feel it’s pretty accurate,” Mr. Pence said. “will it be perfect? No. But when I reach that 67th essay, I’m not accurate that is real either. As a united team, we have been pretty good.”
Because of the push for students to be better writers and meet up with the new Common Core State Standards, teachers are looking forward to new tools to simply help out. Pearson, which can be based in London and new york, is regarded as several companies upgrading its technology in this space, also referred to as artificial intelligence, AI, or machine-reading. New assessments to evaluate deeper move and learning beyond multiple-choice answers are also fueling the demand for software to greatly help automate the scoring of open-ended questions.
Critics contend the program does not do a lot more than count words and so can not replace readers that are human so researchers are working difficult to improve the application algorithms and counter the naysayers.
Even though the technology has been developed primarily by companies in proprietary settings, there is a focus that is new improving it through open-source platforms. New players in the market, such since the startup venture LightSide and edX, the nonprofit enterprise started by Harvard University plus the Massachusetts Institute of Technology, are openly sharing their research. This past year, the William and Flora Hewlett Foundation sponsored an competition that is open-source spur innovation in automated writing assessments that attracted commercial vendors and teams of scientists from around the planet. (The Hewlett Foundation supports coverage of “deeper learning” issues in Education Week.)
“Our company is seeing a lot of collaboration among competitors and folks,” said Michelle Barrett, the director of research systems and analysis for CTB/McGraw-Hill, which produces the Roadmap that is writing for in grades 3-12. “this collaboration that is unprecedented encouraging a lot of discussion and transparency.”
Mark D. Shermis, an education professor in the University of Akron, in Ohio, who supervised the Hewlett contest, said the meeting of top public and researchers that are commercial along side input from a number of fields, could help boost performance associated with the technology. The recommendation from the Hewlett trials is the fact that the automated software be used as a “second reader” to monitor the human readers’ performance or provide additional information about writing, Mr. Shermis said.
“The technology can’t try everything, and nobody is claiming it can,” he said. “But it is a technology which has had a promising future.”
The very first essay-scoring that is automated go back to the early 1970s, but there was clearlyn’t much progress made through to the 1990s with the advent regarding the Internet as well as the ability to store data on hard-disk drives, Mr. Shermis said. More recently, improvements were made when you look at the technology’s power to evaluate language, grammar, mechanics, and magnificence; detect plagiarism; and https://essaywritersite.com supply quantitative and feedback that is qualitative.
The computer programs assign grades to writing samples, sometimes on a scale of just one to 6, in many different areas, from word choice to organization. These products give feedback to help students enhance their writing. Others can grade short answers for content. To save lots of money and time, the technology can be utilized in a variety of ways on formative exercises or summative tests.
The Educational Testing Service first used its e-rater automated-scoring engine for a high-stakes exam in 1999 when it comes to Graduate Management Admission Test, or GMAT, according to David Williamson, a senior research director for assessment innovation when it comes to Princeton, N.J.-based company. It also uses the technology with its Criterion Online Writing Evaluation Service for grades 4-12.
The capabilities changed substantially, evolving from simple rule-based coding to more sophisticated software systems over the years. And statistical techniques from computational linguists, natural language processing, and machine learning have helped develop better methods of identifying certain patterns in writing.
But challenges remain in coming up with a definition that is universal of writing, and in training a computer to know nuances such as “voice.”
In time, with larger sets of information, more experts can identify nuanced aspects of writing and increase the technology, said Mr. Williamson, who is encouraged by the new era of openness in regards to the research.
“It’s a topic that is hot” he said. “There are a lot of researchers and academia and industry looking into this, and that is a good thing.”
In addition to making use of the technology to enhance writing when you look at the classroom, West Virginia employs software that is automated its statewide annual reading language arts assessments for grades 3-11. The state spent some time working with CTB/McGraw-Hill to customize its product and train the engine, using 1000s of papers it has collected, to score the students’ writing based on a prompt that is specific.
“Our company is confident the scoring is extremely accurate,” said Sandra Foster, the lead coordinator of assessment and accountability when you look at the West Virginia education office, who acknowledged facing skepticism initially from teachers. However, many were won over, she said, after a comparability study showed that the accuracy of a teacher that is trained the scoring engine performed much better than two trained teachers. Training involved a hours that are few how exactly to assess the writing rubric. Plus, writing scores have gone up since implementing the technology.
Automated essay scoring is also utilized on the ACT Compass exams for community college placement, this new Pearson General Educational Development tests for a school that is high diploma, as well as other summative tests. But it has not yet yet been embraced because of the College Board for the SAT or perhaps the ACT that is rival college-entrance.
The two consortia delivering the new assessments under the most popular Core State Standards are reviewing machine-grading but never have focused on it.
Jeffrey Nellhaus, the director of policy, research, and design for the Partnership for Assessment of Readiness for College and Careers, or PARCC, really wants to know if the technology is supposed to be a fit that is good its assessment, plus the consortium will be conducting a study based on writing from the first field test to see how the scoring engine performs.
Likewise, Tony Alpert, the chief operating officer for the Smarter Balanced Assessment Consortium, said his consortium will assess the technology carefully.
With his new company LightSide, in Pittsburgh, owner Elijah Mayfield said his data-driven method of writing that is automated sets itself apart from other products in the marketplace.
“that which we are making an effort to do is build a system that instead of correcting errors, finds the strongest and weakest sections of the writing and where to improve,” he said. “It is acting more as a revisionist than a textbook.”
The new software, which can be available on an open-source platform, has been piloted this spring in districts in Pennsylvania and New York.
In higher education, edX has just introduced automated software to grade open-response questions for use by teachers and professors through its free online courses. “One regarding the challenges in the past was that the code and algorithms were not public. These people were viewed as black magic,” said company President Anant Argawal, noting the technology is in an experimental stage. “With edX, we place the code into open source where you are able to observe how it really is done to help us improve it.”
Still, critics of essay-grading software, such as for instance Les Perelman, want academic researchers to have broader use of vendors’ products to guage their merit. Now retired, the previous director of this MIT Writing Across the Curriculum program has studied some of the devices and was able to get a high score from one with an essay of gibberish.
“My principal interest is he said that it doesn’t work. Although the technology has some limited use with grading short answers for content, it relies way too much on counting words and reading an essay requires a deeper degree of analysis best done by a person, contended Mr. Perelman.