Difference between revisions of "2005:Symbolic Melodic Similarity Results"
From MIREX Wiki
(→Result) |
|||
| Line 1: | Line 1: | ||
==Introduction== | ==Introduction== | ||
| − | + | These are the results for the 2005 running of the Audio Melody Extraction task set. | |
| − | + | ===Goal=== | |
| + | To retrieve the most similar incipits from the RISM A/II collection, given one of the incipits as a query. | ||
| + | |||
| + | ===Dataset=== | ||
| + | 558 MusicXML files of the incipits from the RISM collection. 11 incipits of this collection are chosen as queries. These MusicXML files were converted to MIDI files. | ||
==Result== | ==Result== | ||
| Line 23: | Line 27: | ||
|1 | |1 | ||
|[https://www.music-ir.org/mirex/abstracts/2005/grachten.pdf Grachten, Arcos & Mántaras] | |[https://www.music-ir.org/mirex/abstracts/2005/grachten.pdf Grachten, Arcos & Mántaras] | ||
| − | |65.98% | + | |align="right " | 65.98% |
| − | |55.24% | + | |align="right " | 55.24% |
| − | |51.72% | + | |align="right " | 51.72% |
| − | |44.33% | + | |align="right " | 44.33% |
| − | |MIDI | + | |align="center " | MIDI |
| − | |80.174 * | + | |align="right " | 80.174 * |
| − | |B0 | + | |align="right " | B0 |
|[https://www.music-ir.org/mirex/results/2005/sym-melody/GAM_eval.txt GAM_eval.txt] | |[https://www.music-ir.org/mirex/results/2005/sym-melody/GAM_eval.txt GAM_eval.txt] | ||
|---- | |---- | ||
|2 | |2 | ||
|[https://www.music-ir.org/mirex/abstracts/2005/orio.pdf Orio, N.] | |[https://www.music-ir.org/mirex/abstracts/2005/orio.pdf Orio, N.] | ||
| − | |64.96% | + | |align="right " | 64.96% |
| − | |53.35% | + | |align="right " | 53.35% |
| − | |42.96% | + | |align="right " | 42.96% |
| − | |39.86% | + | |align="right " | 39.86% |
| − | |XML | + | |align="center" | XML |
| − | |24.610 | + | |align="right " | 24.610 |
| − | |B4 | + | |align="right " | B4 |
|[https://www.music-ir.org/mirex/results/2005/sym-melody/O_eval.txt O_eval.txt] | |[https://www.music-ir.org/mirex/results/2005/sym-melody/O_eval.txt O_eval.txt] | ||
|---- | |---- | ||
|3 | |3 | ||
|[https://www.music-ir.org/mirex/abstracts/2005/suyoto.pdf Suyoto & Uitdenbogerd] | |[https://www.music-ir.org/mirex/abstracts/2005/suyoto.pdf Suyoto & Uitdenbogerd] | ||
| − | |64.18% | + | |align="right " | 64.18% |
| − | |51.79% | + | |align="right " | 51.79% |
| − | |40.42% | + | |align="right " | 40.42% |
| − | |41.72% | + | |align="right " | 41.72% |
| − | |MIDI | + | |align="center " | MIDI |
| − | |48.133 | + | |align="right " | 48.133 |
| − | |B3 | + | |align="right " | B3 |
|[https://www.music-ir.org/mirex/results/2005/sym-melody/SU_eval.txt SU_eval.txt] | |[https://www.music-ir.org/mirex/results/2005/sym-melody/SU_eval.txt SU_eval.txt] | ||
|---- | |---- | ||
|4 | |4 | ||
|[https://www.music-ir.org/mirex/abstracts/2005/typke.pdf Typke, Wiering & Veltkamp] | |[https://www.music-ir.org/mirex/abstracts/2005/typke.pdf Typke, Wiering & Veltkamp] | ||
| − | |57.09% | + | |align="right " | 57.09% |
| − | |48.17% | + | |align="right " | 48.17% |
| − | |35.64% | + | |align="right " | 35.64% |
| − | |33.46% | + | |align="right " | 33.46% |
| − | |MIDI | + | |align="center " | MIDI |
| − | |51240 | + | |align="right " | 51240 |
| − | |B4 | + | |align="right " | B4 |
|[https://www.music-ir.org/mirex/results/2005/sym-melody/TWV_eval.txt TWV_eval.txt] | |[https://www.music-ir.org/mirex/results/2005/sym-melody/TWV_eval.txt TWV_eval.txt] | ||
|---- | |---- | ||
|5 | |5 | ||
|[https://www.music-ir.org/mirex/abstracts/2005/mikkila.pdf Lemström, Mikkilä, Mäkinen & Ukkonen (P3)] | |[https://www.music-ir.org/mirex/abstracts/2005/mikkila.pdf Lemström, Mikkilä, Mäkinen & Ukkonen (P3)] | ||
| − | |55.82% | + | |align="right " | 55.82% |
| − | |46.56% | + | |align="right " | 46.56% |
| − | |41.40% | + | |align="right " | 41.40% |
| − | |39.18% | + | |align="right " | 39.18% |
| − | |MIDI | + | |align="center " | MIDI |
| − | |10.007 * | + | |align="right " | 10.007 * |
| − | |B0 | + | |align="right " | B0 |
|[https://www.music-ir.org/mirex/results/2005/sym-melody/LMMU_P3_eval.txt LMMU_P3_eval.txt] | |[https://www.music-ir.org/mirex/results/2005/sym-melody/LMMU_P3_eval.txt LMMU_P3_eval.txt] | ||
|---- | |---- | ||
|6 | |6 | ||
|[https://www.music-ir.org/mirex/abstracts/2005/mikkila.pdf Lemström, Mikkilä, Mäkinen & Ukkonen (DP)] | |[https://www.music-ir.org/mirex/abstracts/2005/mikkila.pdf Lemström, Mikkilä, Mäkinen & Ukkonen (DP)] | ||
| − | |54.27% | + | |align="right " | 54.27% |
| − | |47.26% | + | |align="right " | 47.26% |
| − | |39.91% | + | |align="right " | 39.91% |
| − | |36.20% | + | |align="right " | 36.20% |
| − | |MIDI | + | |align="center " | MIDI |
| − | |10.106 * | + | |align="right " | 10.106 * |
| − | |B0 | + | |align="right " | B0 |
|[https://www.music-ir.org/mirex/results/2005/sym-melody/LMMU_DP_eval.txt LMMU_DP_eval.txt] | |[https://www.music-ir.org/mirex/results/2005/sym-melody/LMMU_DP_eval.txt LMMU_DP_eval.txt] | ||
|---- | |---- | ||
|7 | |7 | ||
|[https://www.music-ir.org/mirex/abstracts/2005/frieler.pdf Frieler & Müllensiefen] | |[https://www.music-ir.org/mirex/abstracts/2005/frieler.pdf Frieler & Müllensiefen] | ||
| − | |51.81% | + | |align="right " | 51.81% |
| − | |45.10% | + | |align="right " | 45.10% |
| − | |33.93% | + | |align="right " | 33.93% |
| − | |33.71% | + | |align="right " | 33.71% |
| − | |MIDI | + | |align="center " | MIDI |
| − | |54.593 | + | |align="right " | 54.593 |
| − | |B4 | + | |align="right " | B4 |
|[https://www.music-ir.org/mirex/results/2005/sym-melody/FM_eval.txt FM_eval.txt] | |[https://www.music-ir.org/mirex/results/2005/sym-melody/FM_eval.txt FM_eval.txt] | ||
|---- | |---- | ||
|} | |} | ||
Latest revision as of 16:07, 2 August 2010
Contents
Introduction
These are the results for the 2005 running of the Audio Melody Extraction task set.
Goal
To retrieve the most similar incipits from the RISM A/II collection, given one of the incipits as a query.
Dataset
558 MusicXML files of the incipits from the RISM collection. 11 incipits of this collection are chosen as queries. These MusicXML files were converted to MIDI files.
Result
The scores reported here are averaged over the 11 queries.
| Rank | Participant | Average Dynamic Recall | Normalized Recall at Group Boundaries | Average precision (non-interpolated) | Precision at N documents (N is number of relevant document) | Input Data Format | Runtime (s) | Machine | Detailed Evaluation Result |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Grachten, Arcos & Mántaras | 65.98% | 55.24% | 51.72% | 44.33% | MIDI | 80.174 * | B0 | GAM_eval.txt |
| 2 | Orio, N. | 64.96% | 53.35% | 42.96% | 39.86% | XML | 24.610 | B4 | O_eval.txt |
| 3 | Suyoto & Uitdenbogerd | 64.18% | 51.79% | 40.42% | 41.72% | MIDI | 48.133 | B3 | SU_eval.txt |
| 4 | Typke, Wiering & Veltkamp | 57.09% | 48.17% | 35.64% | 33.46% | MIDI | 51240 | B4 | TWV_eval.txt |
| 5 | Lemström, Mikkilä, Mäkinen & Ukkonen (P3) | 55.82% | 46.56% | 41.40% | 39.18% | MIDI | 10.007 * | B0 | LMMU_P3_eval.txt |
| 6 | Lemström, Mikkilä, Mäkinen & Ukkonen (DP) | 54.27% | 47.26% | 39.91% | 36.20% | MIDI | 10.106 * | B0 | LMMU_DP_eval.txt |
| 7 | Frieler & Müllensiefen | 51.81% | 45.10% | 33.93% | 33.71% | MIDI | 54.593 | B4 | FM_eval.txt |
