Difference between revisions of "2009:Music Recommendation Song Similarity"
From MIREX Wiki
(initial proposal of targeted search subtask) |
m (added new music modifications) |
||
| (One intermediate revision by the same user not shown) | |||
| Line 4: | Line 4: | ||
This subtask deals with targeted recommendation. It deals with tag-based radio, audio search, and prestige-ranked browsing (say by genre). | This subtask deals with targeted recommendation. It deals with tag-based radio, audio search, and prestige-ranked browsing (say by genre). | ||
| − | ===Targeted Search without | + | ===Targeted Search without Personalization=== |
====Source Data==== | ====Source Data==== | ||
| Line 19: | Line 19: | ||
====Execution==== | ====Execution==== | ||
| − | Algorithms recieve the source data and the query playlist. Output is then matched against the ground truth | + | Algorithms recieve the source data and the query playlist. Output is then matched against the ground truth. |
====Evaluation==== | ====Evaluation==== | ||
| + | Evaluate each output against ground truth and combine results. | ||
* Option 1: | * Option 1: | ||
** Pearsons Correlation (order does not matter) against all play-lists in ground truth, best score is reported | ** Pearsons Correlation (order does not matter) against all play-lists in ground truth, best score is reported | ||
| Line 34: | Line 35: | ||
** Pearson correlation | ** Pearson correlation | ||
** F-Measure | ** F-Measure | ||
| + | |||
| + | ===Targeted Search with Personalization=== | ||
| + | This one would be really neat to run, but would require some innovative ground truth... | ||
| + | * user profiles and user playcounts would be added as source data | ||
| + | * ground truth playlists on a topic are hand picked from a group of playlists submitted by a LastFM user | ||
| + | * evaluation is the same as above | ||
| + | |||
| + | ===Targeted Search for New Music=== | ||
| + | Both of the above could remove a random sample of social data, re-run the evaluation using playlists of only removed music, using cross-validation | ||
Latest revision as of 01:33, 5 November 2008
Contents
Song Similarity Subtask
Description
This subtask deals with targeted recommendation. It deals with tag-based radio, audio search, and prestige-ranked browsing (say by genre).
Targeted Search without Personalization
Source Data
- Audio Data
- Song Tags
- Artist Tags
Query
Hand-picked playlist of songs describing a particular constraint
Ground Truth
- Option 1: set of hand-picked ordered playlists
- Option 2: ranked set of songs by play list co-occurence
Execution
Algorithms recieve the source data and the query playlist. Output is then matched against the ground truth.
Evaluation
Evaluate each output against ground truth and combine results.
- Option 1:
- Pearsons Correlation (order does not matter) against all play-lists in ground truth, best score is reported
- Kendall Tau (order matters) same as above
- Option 2: Numeric output (strength of a recommendation)
- Mean error
- Recommendation error
- Option 2: Ordered sets
- ROC area
- Kendall tau
- Option 2: Unordered sets
- Pearson correlation
- F-Measure
Targeted Search with Personalization
This one would be really neat to run, but would require some innovative ground truth...
- user profiles and user playcounts would be added as source data
- ground truth playlists on a topic are hand picked from a group of playlists submitted by a LastFM user
- evaluation is the same as above
Targeted Search for New Music
Both of the above could remove a random sample of social data, re-run the evaluation using playlists of only removed music, using cross-validation
