Magnatune blog

Full text searching added to Magnatune

A full text searching feature has been added to the magnatune web site.

Now, on the home page, is a search field:

I really admire Amazon's search engine, where the results are organized by type of object (books separated from CDs, etc), but that's fairly hard to do with most full text search engines. Amazon wrote their own.

Also, most full text search engine use the HTML page title to display the search hit, but that's often not helpful. For example, if you search for a song name, and you get a page that says "Artist: albumname" you may wonder why there's a hit there. I think that the search engine result should display titles relevant to the type of object, ie "matched a song named X from the album Y by artist Z"

I also like search to be really fast. Lots of sites make you wait 10 seconds or more for results. On tile.net a million-hits-a-day site I used to run (and sold to an Internet.com funded dot-com company), fully 30% of the page hits were searches, and I wrote a search engine for that one (at that time, all in C, using dbm files and AOLServer). I really liked the relevant, organized results that I could provide.

So, I wrote my own search engine for Magnatune. It searches several types of data, and thinks about them differently.

In order of importance, the Magnatune search engine looks at (and groups results by):

1) band name

2) album name (and band name, so a search for a band gives their albums too)

3) collection names (from http://magnatune.com/collections/) and the album names that are in each collection, so if you search for "Artemis" you'll see that they're in 4 different collections (Woman Singing Electro Pop, Chillout, etc...)

4) Song names (with the search hit linking to the album page, which displays gives the song names, and also linking to the artist's page). This will help people find artists and albums that they remember a fragment of a song name of.

5) Artist bios (a search for "Artemis" shows you what other bands their musicians play in)

For an example, search for "Artemis"
http://my.magnatune.com/search?w=artemis

and you'll get these results:

I plan on adding the documents from the "/info/" section into the search index: currently only information about our artists and their music is in there. That's going to take a bit of work, to clean up the HTML titles so that they're consistently meaningful, as I want to make sure I don't return useless search results.

I'm currently showing a maximum of 5 hits from each category, and displaying a "show all" link when not all the results are given. However, the total number of hits that were found is given (ie, "violin" hits on 58 songs and 21 artist bios)

For the geeks out there who want the gory details, here they are:

1) I'm using Berkeley DB as the underlining database. They have a *great* Tcl (my programming language of choice) toolkit that almost exactly matches their C API.

2) Benchmarking: I'm getting 760 full text searches per second, on mini very-slow mac mini, from a for() loop just exercising the search. In the real world, web server and other overhead brings that down to 98 full text searches per second. However, that's plenty fast. I even leave noise words like "the" and "and" in, since they're handled quickly and because "the kokoon" then can find artist names with both ("the" is not noise in an album or artist name)

3) the underlining web application server for the searching is Tclhttpd - the magnatune web site runs on apache. I may switch to AOLserver in the future, as my benchmarks showed 10X faster speeds with mixed HTML/code pages (with Tcl)

4) a separate job creates the indexes, which takes about 3 minutes to build. The web searches work a no-semaphore-lock, read only version of the index files (Berkeley DB really recommends you avoid locking for best performance). Berkeley DB does the caching: I don't cache any results in memory, so that I can handle any size corpus.

5) a simple "inverted index" is used, ie "what pages is this word on? Look up the word, quickly get the pages"

I'm definitely open to feedback and feature requests on this thing. Because it's homemade, I've got a lot of flexibility with what I can do with it.

Why do 63% of our visitors leave right away?

I've always been puzzled by one fact I see in our web logs, namely that 63% of people who visit www.magnatune.com leave within 30 seconds.

Are they coming for some other reason than to listen to music? Do they not like what the home page says? Are they not interested in what the site offers? Is the home page unclear about what's being offered?

Here is how our site visitors break down, based on length of visit:

It's probably fair to guess that all our sales come from the 18% who stay 5 minutes or more. That's not a very efficient use of the visitor traffic that comes to our site!

As I mentioned previously, I'm reading the book "Don't make me think" and it occurred to me that people might not know, from the home page, what the site is going to do for them. They can "explore a genre" -- what does that mean? There are new releases... ok... There's that long paragraph at the bottom, that I'm sure few people read.

"We are not evil, "Internet music without the guilt" and "Magnatune, the open music record label" are probably all too clever for the new visitor, who is wondering "what does this site do? why should I stay?"

Here's what the current top section of the home page currently looks like:

The "internet music without the guilt" slogan is speaking to the guilt you (theoretically) feel by downloading music from peer-to-peer services, a russian download site, or perhaps supporting the (evil) major labels represented on the "legitimate" download services. Problem is, I don't think most people get that, and even if they do, I'm not sure here's that much guilt around. It's just me being too clever.

"The open music record label" is similarly problematic: it refers to "open source" ideals, but that's vague, and it's not clear what the user's payoff is from that, anyway.

So, I'm trying a new approach: clearly saying what the "great reason to stay at this site" is, namely that you can listen to music all you want, and choose to buy stuff once you 100% know you love it. That's reflected in the new home page, that is up as of now:

Hopefully, this change makes it clear that the Magnatune web site is about "listening, and then eventually buying" and that the genres below are to help you choose something to listen to.

I'm going to be watching that 63% number, and seeing if my changes are encouraging more people to stay a few minutes longer.

-john

Don't make me think!

I'm reading the 2nd edition of "Don't make me think!", an excellent web-site usability book.

I got to thinking that the "buy" button at Magnatune doesn't do a very good job of explaining the basics, such as:

1) what am I buying? What does it cost?

2) if I buy a download, what do I get? What formats or quality? Is there copy protection?

3) Do you sell songs or albums or both?

I've revamped the details page in a few ways in improve this:

there are now "buy a download" and "buy a cd" buttons under the artwork. People have asked "how do I buy?" -- the small "buy" link at the end of each album seems to not be seen by people.

Under these buy buttons in a new link to "what you get when you buy" which gives a kind of mini FAQ:

We sell complete albums. Albums are available as a download or as a physical CD. $8 is the price we suggest you pay for an album, but you can choose to pay as little as $5 or as much as $18. The artist gets half of the price you choose. A CD costs $4.97 more than a download. You can download MP3s and perfect-quality WAV files (as well as several other open formats such as FLAC, OGG and AAC). These work on all computers (Windows, Mac OSX & OS9, and Linux) and have no copy protection. Downloads typically take 20 minutes to one hour. When you buy a CD, you also get to download the music right away. You can give 3 copies of the music to your friends.

I wrote in very short sentences to make it punchy and clear. I'd appreciate feedback as to whether I should explain some other things on this page, or if there's something I can remove or that's unclear.

I've also made the "BUY" button bolded and uppercase on the artist pages:


and on the details page the list of choices is in larger text, as one of the tips in the "don't make me think!" book is to use larger text for the most important items:

I'm not too happy that there are three paths now to buying, which are:
1) the "buy" link on the artist and details pages, next to listen hifi/lofi
2) the "buy album" link on the details page
3) the "what you get when you buy" page lets you continue to buy a download or CD

I'm worried that people will wonder "how are these different links different?" but perhaps not, perhaps having several paths will help people find their way to buying. We'll have to see...

I've asked Shannon to rewrite the "choose page" which you get when you hit "buy" so that it answers basic questions better, like the "what you get when you buy" page does.

Experimenting with "featured album" on the home page

We're trying out a new idea, of a "featured album" on the Magnatune home page at http://magnatune.com/

If you go there now, you'll see, on the lower right hand side of the page, a small album cover and description of
Featured Album: Welcome Om by Christopher of the Wolves

The plan is to rotate the featured album regularly. But first, we want to see if the featured album on the home page (making it easy to find something to listen to) increases overall sales, or just directs people to buy more of the featured album.

-john

Location of artists now given at Magnatune

The Magnatune web site now displays the location (in the world) of each musician, under the "details" link:

For the USA, the State and Country is displayed. Outside the USA, the City and Country is displayed. If you click on the location, you are taken to http://magnatune.com/artists/locations where the location of every artist is given:

and you are taken to the part of the page for the artist you just clicked on.

A big reason for doing this are the performing rights collection societies, who have exclusive, and often usurious contracts with their artists.

In the USA, ASCAP and BMI were sued by the government, and under a consent decree must let their musicians be non-exclusive. This means that ASCAP and BMI must recognize agreements directly between a licensor and the music rights holder (in our case, that's the musician, with Magnatune acting as their legally authorized representative and sub-licensor).

This means that organizations licensing music from us in the USA obtain a performing rights royalty waiver (no ASCAP/BMI fees) and it also means that organizations not in the USA can license our music and get a waiver, but only if they exclusively play USA (ASCAP/BMI) artists.

We've had to defend our license before lawyers from PRS/MCPS in London, England, and we won. The key point in that discussion was that non-USA jurisdictions need to keep to musicians from the USA (ASCAP/BMI musicians) and then PRS recognizes our waiver.

I've modified the "public space" license (ie, restaurants, cafes, art galleries) to explain this reality.

The new pages make it easy for people licensing music to stick to Magnatune's USA musicians.

This is what the public space licensing page now says (at the bottom):

A Performing Rights Society (ie, ASCAP / BMI / PRS / GEMA / SESAC) fee waiver is granted with this license so that you do not have to pay any fees to them if you are playing exclusively our music in your public space. If you are in the USA, you may license any music you like. If you are outside of the USA, you should only license music by musicians inside the USA. You can be sure you are safe, as we have successfully defended our music license before lawyers from PRS/MCPS (the UK's performing music rights collections society) and won: the crucial point is that if you are playing our music outside of the USA, you must only use music from USA musicians. Please note that the collection societies will only recognize this waiver if you play our music exclusively, and never mix in unlicensed music from other sources.

Unreliable audio streaming hopefully fixed

I've found and fixed a big problem with the http audio streaming at Magnatune. Several people in Europe had complained about being unable to reliably stream our music.

My own tests found transfer rates of 50k/second from he3.magnatune.com but 3mb/second from www.magnatune.com. Since I know Apache (which is on the www server) is slower than mathopd (which is on the he3 server), either there was a software bug in mathopd or a machine/networking problem.

The problem was that the 100mbit Intel Ethernet card was set to autosense its speed, and wasn't doing that correctly, so we were getting 1/60th the speed we were supposed to get. This new problem appeared few months ago when I moved from http://he4 to http://he3 -- he4 didn't have this problem (he3 had more disk space, which we needed). For the Linux geeks out there, the solution was adding "options e1000 Duplex=2,2 Speed=100,100" to the /etc/modprobe.conf file and rebooting. This is spelled out by Intel docs at http://support.intel.com/support/network/sb/CS-009209.htm#speed_duplex

I'm now getting 5.5M/second transfer rate from http://he3, which is the machine that feeds all our audio streams, and this seems like a more sane speed (I believe that's 44mbits, on a 100mbit line).

Magnatune fans: please let me know if you see an improvement in either drop-outs or time-to-start-playing-the-song.

On a separate topic, I thought the problem might be the "mathopd" web server I'm using, and investigated and benchmarked several high performance alternatives. I finished by using thttpd http://www.acme.com/software/thttpd/thttpd.html because the comparison charts http://www.acme.com/software/thttpd/benchmarks.html they created show mathopd a close running up to them (in comparison, Apache is 1/5th as fast with large files) which agrees well with my own tests 2 years ago, where I found mathopd to be the fastest.

My own use of the "ab" benchmark tool finds thttpd to be about 25% faster than mathopd with a moderate load (10 users) and a mp3-sized file (3megs).

To run my benchmark, thttp took 14 seconds, mathopd took 19 seconds, and apache took 78 seconds. This is why a specialized web server is a useful thing when serving lots of multi-megabyte files.

Shannon's report to our musicians

Shannon has taken over artist relations, and in that role wrote this month's "for our musicians" newsletter which we send to our musicians to keep them up to date with our progress. I'm told people read this blog to "peek behind the curtain" and see how something like Magnatune is run, so I've pasted the main body of her newsletter below:

In other news, we've been *very* active on the conference and trade show circuit over the last month, and have made lots of great connections in the process. As a new member of the Recording Academy, John attended the NAMM show in late January where he met up with Jerry Marotta (drummer to Peter Gabriel, Indigo Girls, and Magnatune artist Sun Palace) among others. After attending two different public radio related conferences over the last month, Teresa Malango -- who handles all the licensing for Magnatune -- is making some serious inroads with National Public Radio's music and podcasting staff, attending two invite-only conferences (in New York and Texas) in the past few weeks. Meanwhile, I represented Magnatune at the fabulously cool Media That Matters Film Festival, where I saw a short film that prominently featured a great tune by Beat Under Control. It's #12 here if you'd like to watch it: http://www.mediathatmattersfest.org/mtm05/, and I also recommend #14, "World on Fire." Couple of other things before we get to John's recent blog entries... First, who's going to South by Southwest? Teresa and I will be there to spread the word among filmmakers that they can license your music, and I recently learned that the fabulous Drop Trio will be perfoming too. If you'll be there as well, please drop me a line so I can see your show and/or just meet up with you and buy you a drink! Second, we'd like to start making Magnatune business cards available to you. It would have the standard Magnatune info on it, as well as your individual Magnatune URL and any other contact information you'd like to share. We want to make sure that the expense of doing this is justified, so if you'd like a box -- please do mention a little bit about why you think you need them and how they'd be of use to you. If you never play out and aren't very active with promoting your music, it may be best to hold off on the business cards until you ramp up a bit. Third, I'm starting to incorporate more press quotes on the artist pages. (i.e.: http://www.magnatune.com/artists/lieberson) If you have some good ones in your collection, please share! I'll post them to your page. Finally, starting this Saturday, we're going to try including a featured album on the Magnatune home page. We're not sure yet if this will be a permanent addition to the web site; for now, we'd just like to try it to see how it goes. You'll be hearing from me on this next month with feedback.

Quoting reviews on magnatune artist pages

As a matter of policy, I've never put reviews on the artist pages I've made for Magnatune artists. The reason is that in the dot-com era I remember reading of a few lawsuits where the one site sued another for copyright infringement, for quoting text from their site w/o permission.

Shannon has taken over artist relations at Magnatune and she has a stronger traditional marketing and press background than I have. She feels strongly that including reviews and favorable quotes helps gain credibility and influences potential listeners & buyers. I told her about my objections to this, and so she decided to research it a bit. This is what she found:

A fabulous quote about Lorraine Hunt Lieberson from The New Yorker heightened my desire to figure out whether or not we can quote these sources on the Magnatune artist pages, so I've been researching the fair use issue over the last few days and have learned that this practice -- in all likelihood -- falls squarely within the parameters of fair use. While it's true that commercial motives aren't given as much leeway as non-commercial ones, we easily meet the vast majority of the other fair use stipulations: --quoting from published works --quoting from factual/non-fiction based works --quoting a small quantity of the whole work --no significant effect on the market or potential market for copyrighted work --no similar product being marketed by the copyright holder Since fair use decisions are never cut and dried, it's impossible to say that this would *guarantee* our right to quote reviews, but since most decisions are based on whether or not someone is meeting the *majority* of the stipulations, it's likely that it's totally fine for us to do this. Case law is on our side as well: In 1983 Reports sued a vacuum cleaner manufacturer to prevent them from using their findings in its advertising. The court ruled that quoting Consumer Reports constituted fair use under 17 U.S.C. S 107. This was even in light of the consideration that the ad *did* to some degree affect CR's market for the particular issue in which the product was reviewed. See also: http://tinyurl.com/8k24z (a Nolo Press article, search on the word "vacuum" to get to the relevant paragraph) I've also emailed a bunch of companies like Powell's Books to see how they address the issue, as I *seriously* doubt that they license content from publishers on product pages like this one: http://tinyurl.com/cp9z7

The Consumer Reports case is interesting, because the Nolo press article does not mention "quoting from a review in an advertisement" as fair use, and this is how a magazine might see the Magnatune artist's pages.

Magnatune 2 years sales chart

The chart below shows Magnatune sales for the past 2 years.

The top line shows over all revenue, whereas the lines below show just download/cd sales, and the lowest line shows licensing. All lines are in dollars, but I'm now showing the absolute dollar amount.

One thing stands out is that we're making most of our revenue from consumer sales, not from licensing.

Another is the overall trend, which is quite positive. We were at a very obvious plateau for 18 months, and then broke out of that in August 2005 with a 2x growth in licensing, and a 40% growth in downloads, which is now 6 months going as of this chart.

Licensing significantly revamped

The music licensing options and questions have been significantly revamped, since I started to get quite a bit of feedback as I got into it and everyone had great ideas for improvements.

Here's a rundown of the changes: it's a long list, so be forewarned!

First, the licensing categories have all been clarified and consolidated down to:

What kind of music license? 1. Film: sync license 2. Single units: wedding video, small quantity for-profit 3. Video: CDROM, VHS, DVD, and for-sale production. 4. Slide show or Powerpoint. 5. TV ad or radio ad (or radio production) 6. Audio projects 7. Internet web site, Flash 8. Podcasts & vblogs 9. Non-Commercial: schools and Creative Commons 10. TV show: sync license 11. Video game and software 12. Sampling: remixes, covers & derivative works 13. Music compilation: CD, DVD and computer audio 14. Public Space: restaurants, trade shows & retail spaces 15. Telephone music on hold 16. Custom bid: other projects and special situations

Each license category explains itself -- previously only the title of the license was given, leading people to wonder "is this the right license for what I'm doing?". For example, for film, it now says:

This license is for any video or film project which will be shown in theaters. If your project will not appear in theaters, a "video" or "single units" license is what you need.

a new slide show or powerpoint license was added, described as:

This license is for adding music to a slide show presentation or a powerpoint document, or a similar kind of computer-based presentation. The finished product must not be available for sale: if you would like to place music on a product for sale, you need a video/cdrom license.

I wanted to get rid of the "corporate" license, because none of us really had any idea what it precisely meant: it is more a type of distribution on another project (a corporate video, or a powerpoint presentation at an annual meeting, for example) that its own type of media. That's why I added the slides/powerpoint license. This also meant a new question needed to be added to the video license:

Distribution: internal use only available to the public

I've often wondered if people are apprehensive about licensing music from us, because we've got a counterculture/revolutionary message (the "we are not evil" thing). After all, when you buy a license, all you're buying is a legal document that keeps you from getting sued.

To try to address this concern, every page now explains exactly what rights you're getting, in more traditional language (mechanical, publishing and synchronization rights) as well as putting our not-evil philosophy into practical terms, and explaining about rights clearances. This is the paragraph under the slide show/powerpoint license:

This license will grant you syncronization and mechanical rights, to place the licensed song in your presentation. You may give away or sell your work, as you see fit. We will not ask to see your work or exercise any artistic control over the use of the music in your work. We don't think it's right for musicians to dictate what a film-maker's vision should be, and we hope you'll honor the flexibility we give you by giving us your business. We see to it that the music has had the appropriate legal rights pre-cleared, and the music license you will receive contains a provision indemnifying you of legal liability in that regard. We have licensed our music to hundreds of projects, including global megacompanies such as the Regal Entertainment Group and Hitachi. You can rest assured that buying a license from us will get you in the clear, and we're happy to physically sign a purchased music license so that you have a hard copy as proof.

Each of these paragraphs is tailored to the kind of license being viewed. For example, the film license contains extra text speaking to film-makers:

If your film does not yet have worldwide distribution, you can purchase the smallest territory license you need, and come back at a later date, to obtain a wider territory music license. Many films have started with the $44 film festival license, and after getting picked up for distribution, purchased a worldwide territory music license (as well as optional DVD and music compilation rights) at our standard published rates. If you need a "step license" that shows the increasing price of a music license at increasingly wider distributions, just create several license agreements for different territories using our forms (each contract ID will be named CM######), and we will honor those price quotes as a step license when/if you come back to us later. You can then use those price quotes with confidence in your proposals to film distributors.

Teresa told me that a lot of the calls she gets are from people with no money who see our prices as too high and want a special deal. We weren't previously accommodating that scenario for one simple reason: I had created all the scenarios and prices by looking at our licensing competitor's price sheets, and none of them were offering low cost home-use and starving-artist licenses. Of course, those competitors don't have automated web-based licensing, so there's no reason for us to not offer these.
Now, a number of use cases have much lower budget options. For example, the Video/DVD license goes down to "under $100 or no budget" and stars at $19, music on hold for a home office starts at $24 for a year, and a no-budget film starts at $25.

To further gain the confidence of licensees, I've added call-out endorsement/quotes for some of our license types, where I could get a quote from someone successful in exactly that field. For example, for a movie, this quote now appears at the bottom of the page:

"As a film producer, it's an absolute pain in the butt to license quality music and your company makes it so painless and easy, I really appreciate that. I will continue to purchase music from you for use in my films and I feel sure that my success will be in part to your company and the music you produce." - Unsolicited endorsement from Quito Washington, award winning film maker and author of "DV Filmmaking Explained"

an "audio projects" license type has been added, described as:

Audio projects This license is for music in audio projects, such as books on tape, motivational tapes, audio guides for art exhibits, commercial podcasts. If your audio project will be publicly broadcast, you should get a "radio production" license instead.

On the "internet" license, I've always asked how many impressions the page is projected to have. This never worked well, since most time when people are licensing music for a web page, it's for a new page or site, and they have no idea how many hits it will get. Worse, if it exceeds what they bought, it's very unlikely they'll come back to us and send us money for an upgraded license. Our competitors ask for this estimate, which is why I had it, but it just doesn't work in the real world.

Now, the internet license asks for the budget of the project or web site, which is a much better indicator of how important this is, and is how film licensing works.

I've removed the "movie trailer" license -- I can't imagine anyone licensing our music for a trailer where the music wasn't also used in the film itself, and the film license allows use in trailers. Again, this is something our competition offers, but we've never sold one and I doubt we ever would, so it's removed to make things simpler.

The film license now includes the concept of "distribution" so that internal-only videos are a much less expensive license. This also makes explicit the idea of videos distributed on the Internet with the "external (such as on web sites or for sale)" choice.

It was our policy that if someone made a song with more than one sample in it, we'd still just charge them the one-sample fee. This is now explicitly stated on the "sampling / remix" license:

This license permits you to use any number of audio samples from a song to create a new song of your own. You can also make remixes, mash-ups or other derivative works. If you use several samples from us in one song, you will only be charged the single-sample price.

Similarly, it was our policy that compilations that used more than 10 songs on a CD would simply pay the 10 song price. This is now explicit on the "music compilation" license. This license also now lets you license several songs (or an entire CD) at once, where previously it was limited to one song, and Teresa needed to get involved for multiple song licenses:

License Music for Music compilation (CD, tape or vinyl) This license allows you to redistribute and resell CDs, DVDs or other media containing the entire song or an entire album. For example, this can be used to make a rebranded CD for sale in your store or through traditional CD distribution, or a promotional giveaway with your company's information on it.

People have always asked what the relationship was between our public space license and ASCAP/BMI. Previously, I was avoiding a direct confrontation with these entities, but BMI challenged us over our Internet radio stations (we won) and PRS in the UK went after us for a performance-royalty free license to a major home-improvement store chain that was using our music (we won again). So, I feel confident now in licensing music with a performance-fee waiver, and this is made explicit in the public space license:

License Music for Public Space (ie: Restaurant, Trade Shows Art Gallery) This license allows you to play an album of our music in a public space. An ASCAP/BMI fee waiver is granted with this license so that you do not have to pay any fees to them if you are playing our music in the USA. This right is well established in USA case law by the "consent decree" under which ASCAP and BMI operate. We can supply non-USA jurisdictions with performance-royalty-free music (about half our music is in this category): email us if you need this.

Note that this does mean that a musician licensed in this way wouldn't collect ASCAP/BMI fees from this licensor's playing of the music, but that should be more than offset by the fact that they get 1/2 the fee we charge, which would be way more than a musician would see out of the ASCAP/BMI machine.

and the "custom bid" item is new, and much friendly in inviting a dialog for those cases where the automated system doesn't work.