Crowd workers, AI make conversational agents smarter
Evorus is a chatbot that uses a combination of artificial intelligence and crowd workers to answer questions on a broad range of topics.
Conversational agents such as Siri, Alexa and Cortana are great at giving you the weather, but are flummoxed when asked for unusual information, or follow-up questions. By adding humans to the loop, Carnegie Mellon University researchers have created a conversational agent that is tough to stump.
The chatbot system, called Evorus, is not the first to use human brainpower to answer a broad range of questions. What sets it apart, says Jeff Bigham, associate professor in the Human-Computer Interaction Institute, is that humans are simultaneously training the system’s artificial intelligence, making it gradually less dependent on people.
Like an earlier CMU agent called Chorus, Evorus recruits crowd workers on demand from Amazon Mechanical Turk to answer questions from users, with the crowd workers voting on the best answer. Evorus also keeps track of questions asked and answered and, over time, begins to suggest these answers for subsequent questions. The researchers also have developed a process by which the AI can help to approve a message with less crowd worker involvement.
“Companies have put a lot of effort into teaching people how to talk to conversational agents, given the devices’ limited command of speech and topics,” Bigham said. “Now, we’re letting people speak more freely and it’s the agent that must learn to accommodate them.”
The system isn’t in its final form, but it is available for download and use by anyone willing to be part of the research effort: http://talkingtothecrowd.org/.
A research paper on Evorus, already available online, will be presented by Bigham’s research team later this year at CHI 2018, the Conference on Human Factors in Computing Systems in Montreal.
Totally automated conversational agents can do well answering simple, common questions and commands and can converse in depth when the subject is relatively narrow, such as advising on bus schedules. Systems with people in the loop can answer a wide variety of questions, Bigham said, but with the exception of concierge or travel services for which users are willing to pay — agents that depend on humans are too expensive to be scaled up for wide use. A session on Chorus costs an average of $2.48.
“With Evorus, we’ve hit a sweet spot in the collaboration between the machine and the crowd,” Bigham said. The hope is that as the system grows, the AI is able to handle an increasing percentage of questions, while the number of crowd workers necessary to respond to “long tail” questions will remain relatively constant.
Keeping humans in the loop also reduces the risk that malicious users will manipulate the conversational agent inappropriately, as occurred when Microsoft briefly deployed its Tay chatbot in 2016, said Ting-Hao Huang, a Ph.D. student in the Language Technologies Institute (LTI). Huang developed Evorus with Bigham and Joseph Chee Chang, also a Ph.D. student in LTI.
During Evorus’ five-month deployment with 80 users and 181 conversations, automated responses to questions were chosen 12 percent of the time, crowd voting was reduced by almost 14 percent and the cost of crowd work for each reply to a user’s message dropped by 33 percent.
Evorus is a text chatbot, but is deployed via Google Hangouts, which can accommodate voice input, as well as access from computers, phones and smartwatches. To enhance its scalability, Evorus uses a software architecture that can accept automated question-answering components developed by third parties.
This research is supported by Project InMind, a Carnegie Mellon effort sponsored by Yahoo!/Oath to develop advanced technologies for personalized digital assistants.
Edited by Ariasun editorial board
for ariasun products online shopping ,click here.
website ariasuniweb net ariasuniportal ariasuni Health Journal Family physician Medical Health Ariasan Elcomp </ em> Elcomp Exhibition </ em> Online Elcomp Therapy Health Electronics Tourism Aryans City Net Soocom Ariasun Soopak Ariasun medipharma Advertisement Reader The requirements Employment Currency Exchange Currency Coin Gold < em> car property land building stock exchange bank Government People Leader Assembly Quran Book Student University Knowledge base Exercise Sima Sound Religion < / em> Life People Hope Work Dread Unemployment youth exercise information technology computer < / em> Hardware Software Computer Server Virtual Tablet < / em> Mobile Zanjan Tehran Iran Germany Greece Georgia Georgia Dubai World Air Space < em> Country Product Finder AriaSun International Group Online Store WikiPharma AriaSun Production Industrial Operations Equip Installation & Investment Air Space International Trading Research Industrial Business Medical Dummy Undoubtedly Health Drug Therapy Health Information Technology & Computer Airlines Tourism Logo Design Sticker Design Advertising Campaign Science & Research News </em > Political News Economic Trade International Stock Market News Medical Tourism & Cultural Heritage Aeronautics & Tourism Sport & Olympic Domain and University The Quran Arts & Culture Industry & Technology Accidents Ladies Currency and Gold Coins Flats Rip’s Web Site Kandasheh Ahwaz Web Site Foot Website Abuse Clean Sean Cinere Live Nature Whats Punababy Mussels Clinic Radarians Man AriaSun Preddies Reservations < Em> Tourists Unemployment youth exercise CityNet PakSaman SooComAriaSun SooPakAriaSun MediPharma AriaSun Kalayab AriaSunBazar AriaSunBazar Medeca MediKala Artists Group AriaSunGroup Web site designing RP Pilgrimage website Historical tours website Web Resident Resorts site Tours website The Pana The Natural Tours Website Kandesheh Ahwaz The Scientific Tour Website Oil & Refineries The Sports Tour Website Gas The Web Tours Water Exposure Tour Site Travel Tour Website Travel Tours The Last-minute Tours Site Tours of the last minute Web-site of the local tours Recreation site Internal tours site Pilgrimage Fun Places Design Web Design City Net Designing a Holiday Resort Pilgrimage Pilgrimage Website designing pilgrimage City Net Pilgrimage Travel </em > Designing Travel Planner Website Designing Travel Places Historic Places Designing Historic Places Historic Places Design Residences Residential Residential Design Residential Residential Design Cultural Places Designing a Web Site for Cultural Places Designing Cultural Places Natural Places Natural Places Designing a natural site Designing a natural website scientific Designing for scientific Designing for scientific sports sports designing sports < Em> Sports Facilities Design Sports Facilities Sport Sport Designing a Sports Center for Extreme Sports Creating places Designing for Exhibition Exhibit places Designing Exhibit Places Designing Exposure Website Web Recreation Recreation Web Design Recreational tours The design of the tourist resorts Pilgrimage Pilgrimage Pilgrimage Pilgrimage Pilgrimage Tours Historic Tours Tour Design Historic Tours Web Tour Design Historical Tours Residential Tours Resorts Tour Design Residential Tour Website Design Cultural Design Cultural Tour Site Cultural Tour Website Natural Designing Natural Tours Designing Natural Tours Web Site Science Designing Scientific Tours Designing Sci-Fi Web Site Designing a Sports Tour Designing a Sports Tour Website Designing a Tours Tour Designing a Touring Web Site Exhibit Exhibit Tour Travel Designing Tours Tour The Last Moment Toure The Last Moment Designing the Last Minute Tours Internal Tours Designing Local Tours Designing a Web Site Tour </em > Foreign Tours Designing a Tours Site Designing a Website for Tours Coastal Work < em> Designing the Coast Guided Tour Urban Designing Urban Tours Urban Touring Sights < Em> Sightseeing Design Sightseeing Web Site Mountaineering Sailing Designing Sailing Tours < Em> Website Designing Sailing Tours Aviation Designing Tours Cruising Cruising ships Creating cruise ship cruises Creating Cruise Cruises Inbound Flights Designing Inbound Flights Foreign Flights Designing a Foreign Airmail Web site Last Minute Flights Web Design Last Minute Deals cheap Website Design Cheap Flights Online Shopping Online Sales Online Selling Website Design Online Shopping Online Store Online Store Design Online Tickets Design an Online Ticket Internet Designing a Web Store Internet Tickets Designing an Internet Ticket > parscoders
Finance & Administration
Soopak Ariasun ( IT )
Sookam Ariasun ( Medical and Veterinary )
Aerospace ( Pad Ariasun )
Manufacture and production ( Rad Ariasun )
Airline and Tourism ( Man Ariasun )
International Commerce ( Mad Ariasun )
Research Center Ariasun ( R&D&C )
Cultural,Satellite and Digital
AriaSun International Group