{"592685":{"#nid":"592685","#data":{"type":"news","title":"Robot Uses Deep Learning and Big Data to Write and Play its Own Music","body":[{"value":"\u003Cp\u003EA marimba-playing robot with four arms and eight sticks is writing and playing its own compositions in a lab at the Georgia Institute of Technology. The pieces are generated using artificial intelligence and deep learning.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EResearchers fed the robot nearly 5,000 complete songs \u0026mdash; from Beethoven to the Beatles to Lady Gaga to Miles Davis \u0026mdash; and more than 2 million motifs, riffs and licks of music. Aside from giving the machine a seed, or the first four measures to use as a starting point, no humans are involved in either the composition or the performance of the music.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe first two compositions are roughly 30 seconds in length. The robot, named Shimon, can be seen and heard playing them \u003Ca href=\u0022https:\/\/www.youtube.com\/watch?v=j82nYLOnKtM\u0022\u003Ehere\u003C\/a\u003E and \u003Ca href=\u0022https:\/\/www.youtube.com\/watch?v=6MSk5PP9KUA\u0022\u003Ehere\u003C\/a\u003E.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EPh.D. student Mason Bretan is the man behind the machine. He\u0026rsquo;s worked with Shimon for seven years, enabling it to \u0026ldquo;listen\u0026rdquo; to music played by humans and improvise over pre-composed chord progressions. Now Shimon is a solo composer for the first time, generating the melody and harmonic structure on its own.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;Once Shimon learns the four measures we provide, it creates its own sequence of concepts and composes its own piece,\u0026rdquo; said Bretan, who will receive his doctorate in music technology this summer at Georgia Tech. \u0026ldquo;Shimon\u0026rsquo;s compositions represent how music sounds and looks when a robot uses deep neural networks to learn everything it knows about music from millions of human-made segments.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EBretan says this is the first time a robot has used deep learning to create music. And unlike its days of improvising, when it played monophonically, Shimon is able to play harmonies and chords. It\u0026rsquo;s also thinking much more like a human musician, focusing less on the next note, as it did before, and more on the overall structure of the composition. \u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;When we play or listen to music, we don\u0026rsquo;t think about the next note and only that next note,\u0026rdquo; said Bretan. \u0026ldquo;An artist has a bigger idea of what he or she is trying to achieve within the next few measures or later in the piece. Shimon is now coming up with higher-level musical semantics. Rather than thinking note by note, it has a larger idea of what it wants to play as a whole.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EShimon was created by Bretan\u0026rsquo;s advisor, Gil Weinberg, director of Georgia Tech\u0026rsquo;s Center for Music Technology.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;This is a leap in Shimon\u0026rsquo;s musical quality because it\u0026rsquo;s using deep learning to create a more structured and coherent composition,\u0026rdquo; said Weinberg, a professor in the School of Music. \u0026ldquo;We want to explore whether robots could become musically creative and generate new music that we humans could find beautiful, inspiring and strange.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EShimon will create more pieces in the future. As long as the researchers feed it a different seed, the robot will produce something different each time \u0026mdash; music that the researchers can\u0026rsquo;t predict. In the first piece, Bretan fed Shimon a melody comprised of eighth notes. It received a sixteenth note melody the second time, which influenced it to generate faster note sequences.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EBretan acknowledges that he can\u0026rsquo;t pick out individual songs that Shimon is referencing. He is able to recognize classical chord progression and influences of artists, such as Mozart, for example.\u003Cbr \/\u003E\r\n\u003Cbr \/\u003E\r\n\u0026ldquo;They sound like a fusion of jazz and classical,\u0026rdquo; said Bretan, who plays the keyboards and guitar in his free time. \u0026ldquo;I definitely hear more classical, especially in the harmony. But then I hear chromatic moving steps in the first piece \u0026mdash; that\u0026rsquo;s definitely something you hear in jazz.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EShimon\u0026rsquo;s debut as a solo composer was featured in a video clip in the Consumer Electronic Show (CES) keynote and will have its first live performance at the \u003Ca href=\u0022https:\/\/www.aspenideas.org\/\u0022\u003EAspen Ideas Festival\u003C\/a\u003E at the end of June. It\u0026rsquo;s the latest project within Weinberg\u0026rsquo;s lab. He and his students have also created a \u003Ca href=\u0022http:\/\/www.news.gatech.edu\/2014\/03\/05\/robotic-prosthesis-turns-drummer-three-armed-cyborg\u0022\u003Erobotic prosthesis for a drummer\u003C\/a\u003E, a \u003Ca href=\u0022http:\/\/www.news.gatech.edu\/2016\/02\/17\/wearable-robot-transforms-musicians-three-armed-drummers\u0022\u003Erobotic third arm for all drummers\u003C\/a\u003E, and an \u003Ca href=\u0022https:\/\/www.youtube.com\/watch?v=3ShaUMM0H-g\u0022\u003Einteractive robotic companion that plays music from a phone and dances to the beat\u003C\/a\u003E.\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":[{"value":"Compositions created using database of well-known pop, classical and jazz artists"}],"field_summary":[{"value":"\u003Cp\u003EResearchers fed a robot nearly 5,000 complete songs \u0026mdash; from Beethoven to the Beatles to Lady Gaga to Miles Davis \u0026mdash; and more than 2 million motifs, riffs and licks of music. The four-armed, marimba-playing machine is using deep learning to write and play its own music.\u003C\/p\u003E\r\n","format":"limited_html"}],"field_summary_sentence":[{"value":"A marimba-playing robot with four arms and eight sticks is writing and playing its own compositions in the School of Music."}],"uid":"27560","created_gmt":"2017-06-13 20:58:17","changed_gmt":"2017-06-13 20:58:17","author":"Jason Maderer","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2017-06-13T00:00:00-04:00","iso_date":"2017-06-13T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"592682":{"id":"592682","type":"image","title":"Shimon  ","body":null,"created":"1497386963","gmt_created":"2017-06-13 20:49:23","changed":"1497386963","gmt_changed":"2017-06-13 20:49:23","alt":"Shimon","file":{"fid":"225900","name":"10C2064-P1-005.jpg","image_path":"\/sites\/default\/files\/images\/10C2064-P1-005.jpg","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/images\/10C2064-P1-005.jpg","mime":"image\/jpeg","size":317975,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/10C2064-P1-005.jpg?itok=73MBV4o_"}},"592683":{"id":"592683","type":"image","title":"Shimon, Musical Robot","body":null,"created":"1497387116","gmt_created":"2017-06-13 20:51:56","changed":"1497387116","gmt_changed":"2017-06-13 20:51:56","alt":"Shimon ","file":{"fid":"225901","name":"10C2064-P1-039.jpg","image_path":"\/sites\/default\/files\/images\/10C2064-P1-039.jpg","image_full_path":"http:\/\/hg.gatech.edu\/\/sites\/default\/files\/images\/10C2064-P1-039.jpg","mime":"image\/jpeg","size":240254,"path_740":"http:\/\/hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/10C2064-P1-039.jpg?itok=uQnCaABo"}}},"media_ids":["592682","592683"],"related_links":[{"url":"http:\/\/www.gtcmt.gatech.edu\/","title":"Center for Music Technology"},{"url":"http:\/\/www.news.gatech.edu\/2014\/03\/05\/robotic-prosthesis-turns-drummer-three-armed-cyborg","title":"Robotic Prosthesis for Drummers"},{"url":"http:\/\/www.news.gatech.edu\/2016\/02\/17\/wearable-robot-transforms-musicians-three-armed-drummers","title":"Robotic Third Arm for All Drummers"}],"groups":[{"id":"1214","name":"News Room"},{"id":"1221","name":"College of Design"},{"id":"60381","name":"CMT - Center for Music Technology"},{"id":"1227","name":"School of Music"}],"categories":[{"id":"135","name":"Research"},{"id":"152","name":"Robotics"}],"keywords":[{"id":"169304","name":"Shimon"},{"id":"1356","name":"robot"},{"id":"167096","name":"school of music"},{"id":"1939","name":"Gil Weinberg"}],"core_research_areas":[{"id":"39521","name":"Robotics"}],"news_room_topics":[{"id":"71881","name":"Science and Technology"},{"id":"71901","name":"Society and Culture"}],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EJason Maderer\u003Cbr \/\u003E\r\nNational Media Relations\u003Cbr \/\u003E\r\nmaderer@gatech.edu\u003Cbr \/\u003E\r\n404-660-2926\u003C\/p\u003E\r\n","format":"limited_html"}],"email":["maderer@gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}