Hello Rocksmith fans!
In February I was in San Francisco for work so I decided to inquire if I could visit the ‘ol Rocksmith studio! Not only was I able to visit after not being there since 2018, but I was also given the opportunity to interview Collin Chu and Lucas Walker about Rocksmith+ Piano.
Elliott: So, Rocksmith+ Piano was a “thing”, they were hiring, and you answered the call. What was the feeling of what you were getting into at the time?
Collin: I had already known about Rocksmith (2014) at the time, it was only for Guitar and Bass. At the time I knew that Rocksmith was adding another instrument (Piano) and I should mention Synthesizer, since it’s part of the product. I envisioned myself creating the curriculum to teach the players how to accomplish that (learn piano) with a gamified approach and not only piano but synthesizer as well. Because, as you can see (from the last two months of content we released) for instance in Linkin Park, it’s not just piano that we chart, but also the synth sections. So that’s what I was envisioning, not just Piano but all instruments, like there’s even a song that has accordion! By December my vision had become a reality.
Elliott: Thanks for the very detailed answer. What is your background when it comes to Piano and education? It also seems like you might be a bit of a gear enthusiast as well!
Collin: I have been playing Piano since I was five years old, much like many other kids back then. In college I studied piano and management during my undergrad. Following that I got two master degrees in music, because I knew it was my passion and that this is what I wanted to do. Before Ubisoft, I was mainly a Piano teacher and also a composer for various gigs like writing jingles, film, indie games and I’ve also maintained being a professor even now in the evenings…
Elliott: Oh wow! You certainly keep yourself busy.
Collin: On top of creating the curriculum and teachings for Rocksmith+ Piano, I also compose most of the music for the Piano side, and at the same time I also lead the Piano transcription team which plays into my management background. A lot of times we have to discuss as a team how best to transcribe certain sections of material, making sure the player is engaged and still having fun instead of holding a single sustained chord. That’s where my background with synth as well as Piano really comes in.
Elliott: ah, so sometimes you have to identify the most engaging part of a song to transcribe between Piano and Synth
Collin: Exactly, it’s still note for note in most cases but when it’s happening at the same time, you only have two hands *laughs*
Elliott: Lucas, what kind of design challenges did you have to tackle when approaching the idea of teaching Piano within the space of the Rocksmith+ visual?
Lucas: The Noteway actually as you see, just with the piano and a lot of the community’s response to it, the Rocksmith Noteway really works well with the format of a Piano, so one of the biggest challenges we had to overcome was a way to represent the repeating pattern of keys and octaves. What we discovered was it was very easy to get lost when it came to where your hands were supposed to go, so we came up with a term while we were in the early prototyping stages called, User Orientation. The question was, what elements could we provide in our design that no matter what you’re doing, or where you’re hands are you always know where you’re at? For instance if you press a key and a lane highlights and you’re outside of an octave you’re supposed to be playing, there are arrows guiding you back to where you should be.
These were all things we were working on before we even had any of the music team onboard, we were already fully prototyped and once Collin came on board we proceeded to figure out how we can gamify elements of notation like staccato, or a roll, there was a lot of back and forth when it came to how we can still stick to traditional music notation while at the same time having a more accessible Rocksmith approach. There was a lot of time spent seeing what works, I have a musical background but it’s more of yelling into a microphone and punching my guitar.
Elliott: Niiiice.
Lucas: I’ve been playing and touring in bands my whole life and stuff like that… I was not a person that could read sheet music very well, so it was a great learning experience for me and I’ve been playing Piano and Organ in my spare time (as a hobby) but never in such an educational way. The real challenge is it’s really cool how we can see the Rocksmith method and the gamification created a low bar of entry for new players (to Piano) and then put a real authentic musical notation on top of it. That was a challenge, but also super exciting.
In summation: An experience that was gamified yet authentic to music transcription.
Elliott: Do you think there’s a fine line of information overload you have to manage with something like Piano, to avoid overwhelming the player? When you look at where we are at and people are asking for more, do you think it’s a challenge to balance that?
Lucas: We definitely plan to constantly improve the Noteway experience with options and tunabilities. We’ve tried to provide on launch as many as those tuneables that we could to provide secondary information or take away to give a cleaner approach. Adding more is always a question of that cognitive overload, that visual noise that could become a rainbow colour confetti explosion. We already provide the option to customize the colours of the keys to your preference, but it’s always a concern. Our very early concepts were very loud, and noisy and were unusable in some ways, so we had to learn from that going forward.
I’ll give you an example of the sustain pedal. The early version was similar to competitors where it was just a badge in the upper corner of the screen that showed you when to press and depress the pedal. Working with Collin we tried to figure out “how do we teach music notation on the Noteway?” so what we actually did with the spatial time something the Rocksmith Noteway is really good at visually capturing.
Elliott: For sure.
Lucas: That’s one of its (Rocksmith+ Noteways) strong cases, so we actually took the musical notation and put them in the side brackets which you see in your peripheral vision, so we are adding that musical notation without distracting you by making you look at the upper corners of the screen. It’s trying to find tricks like that and understanding where the user’s eye needs to look to get the most information, and trying to (very strategically) populate that space, without overwhelming. It is a challenge, but a fun one.
The most exciting thing about Rocksmith+ as a whole, is being in a 3D Space and the interactivity between a game and your analog instrument, that marriage between analog and digital brings up some fun challenges for designers and people who want to do User Experience channels.
Elliott: Agreed, it’s a great product and we are all excited to see how it grows from here. Collin, in terms of transcription what are some challenges you run into when it comes to providing an authentic Piano/Synth experience?
Collin: We definitely came across challenges when it came to Synth and arpeggiators, we had to have discussions on the best way to notate that, to play in an authentic way.
Elliott: Like something, along the lines of Sandstorm.
Collin: Yes, we had a big discussion about that, and how we should transcribe it. When you play that through an arpeggiator it’s just a sustain with 32nd notes, and that’s not how the player is playing in the recording so we probably shouldn’t transcribe it as a sustain, because we want to keep the authentic way to play. When we transcribe a piece of music, we aim to have the playable part from the licensed song be as exact as possible. We want players to feel like they’re that superstar playing that song specifically. So everything is very authentic.
Christopher Woo: I’d also like to mention that we also have Simple Piano arrangements that, while rooted in authenticity, may not provide all the playable parts of a song so players should seek out the Complete arrangements if they’re looking for the most authentic experience.
Elliott: Was there any specific artist that when exploring their material surprised you?
Collin: In terms of difficulty?
Elliott: Exactly.
Collin: If you’re curious about the most challenging songs in the Rocksmith+ catalog there are two types: Jazz… That’s number one, yeah… Those are for REALLY advanced players. Number two are the classical songs, generally speaking they are much harder than pop music, we feature a sizeable amount of them. Those ones are definitely harder to notetrack.
One thing to add onto that, for all the classical songs, we actually had team members create a MIDI mockup that sounds extremely realistic. We have our own way to do it on the team internally. First we tempo map the song, and we adjust the velocity and expressions and do mixing techniques and play on the part via MIDI, and we have back and forth on how we can improve that mockout and the end result is something that sounds extremely real… So if you have a chance to play the classical arrangements they are actually done in MIDI it sounds extremely realistic, I’m very proud about that and only a small amount of our team did work on that specifically.
Christopher Woo: What artists specifically would you say are the most challenging Collin?
Collin: Snarky Puppy? *laughs* Snarky Puppy is more 8/10 but most Jazz is 10/10! Klazz Brothers & Cuba Percussion. that’s a REALLY tough one!
Elliott: Were there any transcriptions you did that made you more of a fan of an artist than before?
Collin: Great Big World comes to mind, Foreigner, I didn’t really listen to them too much. Also, Alicia Keys… I didn’t really listen, I knew of her but when I started working on her material I became a fan. The same thing happened with my team, they’ve definitely grown to appreciate some of the music they work on more than before they started. Alicia Keys, definitely though.
Elliott: How about you Lucas, did you experience that also? For me, I’m a renowned hater of Linkin Park *everyone laughs* when I played their music on Piano, I was like “Oh wait, this is actually really fun,” and also it’s starting to make me enjoy Coldplay a bit more than I’m comfortable with…
Lucas: Yeah, sort of similar my choice of music doesn’t really lie within the realms of artists like Coldplay or Linkin Park for the most part. I did play “In The End” which was really fun and Coldplay “Clocks” I played a lot of that and also [redacted]
Christopher Woo: Nope
Lucas: Oh okay, I guess you have to cut that out, sorry we play a lot of songs it’s hard to keep track *laughs*
Elliott: I’ve been trying to play that on my own!
Lucas: I didn’t think I’d have as much fun as I have with the Ubisoft music, but that Assassin’s Creed Black Flag soundtrack piece with the orchestral part, it’s so solid. I always gravitate more to blues, classical or rock, but I’ve gotten into more of the Pop songs. My wife has caught me playing some songs while working on the game and has done a double take “this is not you!”
I think Piano opens up the door for a lot of musical exploration because of the barrier of entry with Simple Arrangements really helps you get into the song more and it becomes more like a Rhythm Game in some ways. Because it’s so linear, you can just press buttons of your controller, and it helps the user get into that sort of goosebump feeling of interacting with music in a meaningful way. I think because of that it really helps someone explore music outside of their taste while still feeling that sort of gratification with interacting with music they are passionate about.
Elliott: I definitely didn’t expect to be adding a song by The Script to my Spotify playlist, that’s for sure… Collin, Do you think when people start piano they should go straight to an 88 Key Keyboard?
Collin: Everyone has a different opinion on this, but for me personally, I think it’s okay to start on 88. The reason being, in a keyboard/acoustic piano you see the logo in the middle and that’s where Middle C is. So if you use that as a starting point it makes it a lot easier not to get lost.
Elliott: How about you Lucas, do you have an opinion on this?
Lucas: When making our first concepts for the prototype, there were all these considerations we took in. Like what are the most popular key ranges? Like the Middle C marker that he (Collin) is referring to was something we put in because we knew Middle C was there, you could put a sticker on your keyboard.
Before Collin joined the team we had a lot of awesome musicians, pianists that were working on the design team with. I was getting a lot of this information with our other really talented designers to try to come up with the visuals needed to sort of tell this story, you know, to give the markers where your hands were supposed to go.When I first started doing, like the competitor analysis, understanding what these software programs teach, I started on a 49 key keyboard. And it was easy for me to pick up where competitor brands probably show it. Most of their songs sit within that sort of 49 key range and it’s like one of the things that are our VP Nao (Higo) said was like, “we want the full experience, the Cadillac experience.” Yeah. So 49 is not going to be it. “We’re about songs, so you do what you guys need to do to design it so it’s accessible to everyone,” you know?
Elliott: I actually think that when piano was first announced through a Sports Illustrated article, a lot of people were assuming, “so it’s going to be like Rock Band 3 where it charts one hand.” I was like, No, they’re saying it’s going to be 88 Keys fully charted “yeah, now you have my attention”.
Lucas: Our main objective, like I said, is like when we were sitting down and I was just kind of going over with them and I was in my sketchbook is like, “This is the Cadillac,” you know, this is the thing that has all of the elements.
Elliott: What are some challenges you ran into when designing the lessons for piano? What improvements can you see being made to help people in the future?
Collin: One of the challenges is that when people learn piano from private instructors it’s usually based upon classical music and foundations taught in a very specific way. With Rocksmith+ we feature many different styles of music and we had to design a curriculum to support that. The main challenge for designing lessons is we had to analyze all the transcriptions we had as a team and then come up with ways to get the player closer and closer to there. Currently we are in the process of adding even more lessons to help players reach that goal. Approaching education in that way has definitely been a challenge moving away from a traditional classical way.
Elliott: Is the aim in the future (maybe, possibly) to mirror what’s going on with Guitar and Bass lessons when it comes to having lessons in certain styles or genres? For example a House Music type of Club lead
Collin: Yes, that is something we are working on right now, that kind of playing style, core technique, hand technique types of lessons (we have some hand technique lessons right now) but we are planning to do more to help the players get to that level.
Christopher Woo: Just like anything else, we are very interested in community feedback and if we see interest in something like Piano Style Lessons then that’s something we will look into providing. We are always keeping tabs on the ongoing conversations about Rocksmith+ Piano and Rocksmith+ in general.
Elliott: Collin, is there any advice you have for people learning Piano?
Collin: Yeah, try Rocksmith+ because *everyone laughs* I’ll give you a couple reasons why:
You can learn in a gamified approach, if you actually play the song correctly after you practice, those techniques like Staccato, Accent, Pedal, Glissando, Tremolo, it’s actually there thanks to the wonderful design from Lucas and his team. All the articulations that you see in the actual sheet music you see in the Noteway coming at you, we don’t skip it. So if you really put the effort practicing, you aren’t just only fun with a rhythm game, but you should be playing it the exact way it’s supposed to be.
You’re playing the exact part from the artist, and in my team we don’t skip details. I’m very strict on my team when it comes to that.
Christopher Woo: This is Collin Chu, Piano Teacher and Music Professor speaking *laughs*
Collin: I’m actually very nice in general, but when it comes to review, my teams work. I’m actually very strict. If I see something off colour… I go up to them like
“We should get into a meeting and fix this before we ship this out.”
This interview was conducted on February 14th, 2024.
