2009:Music Recommendation Song Similarity

Song Similarity Subtask

Description

This subtask deals with targeted recommendation. It deals with tag-based radio, audio search, and prestige-ranked browsing (say by genre).

Targeted Search without Personalization

Source Data

• Audio Data
• Song Tags
• Artist Tags

Query

Hand-picked playlist of songs describing a particular constraint

Ground Truth

• Option 1: set of hand-picked ordered playlists
• Option 2: ranked set of songs by play list co-occurence

Execution

Algorithms recieve the source data and the query playlist. Output is then matched against the ground truth.

Evaluation

Evaluate each output against ground truth and combine results.

• Option 1:
  • Pearsons Correlation (order does not matter) against all play-lists in ground truth, best score is reported
  • Kendall Tau (order matters) same as above
• Option 2: Numeric output (strength of a recommendation)
  • Mean error
  • Recommendation error
• Option 2: Ordered sets
  • ROC area
  • Kendall tau
• Option 2: Unordered sets
  • Pearson correlation
  • F-Measure

Targeted Search with Personalization

This one would be really neat to run, but would require some innovative ground truth...

• user profiles and user playcounts would be added as source data
• ground truth playlists on a topic are hand picked from a group of playlists submitted by a LastFM user
• evaluation is the same as above

Targeted Search for New Music

Both of the above could remove a random sample of social data, re-run the evaluation using playlists of only removed music, using cross-validation