Difference between revisions of "2006:Score Following Results"
From MIREX Wiki
IMIRSELBot (talk | contribs) m (Robot: Automated text replacement (-http://www.music-ir.org/evaluation/MIREX/2006_abstracts/ +http://www.music-ir.org/mirex/abstracts/2006/)) |
|||
| (3 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
[[Category: Results]] | [[Category: Results]] | ||
==Introduction== | ==Introduction== | ||
| − | These are the results for the 2006 running of the Score Following task set. For background information about this task set please refer to the [[Score Following]] page. | + | These are the results for the 2006 running of the Score Following task set. For background information about this task set please refer to the [[2006:Score Following]] page. |
The aim of the Score Following task is the real-time alignment of incoming music signals to their music score. In this task, 46 recordings and their corresponding MIDI representations of the score were used in the evaluation. These 46 excerpts were extracted from 4 distinct musical pieces. The algorithms were evaluated on their precision in matching events in the audio to the score. | The aim of the Score Following task is the real-time alignment of incoming music signals to their music score. In this task, 46 recordings and their corresponding MIDI representations of the score were used in the evaluation. These 46 excerpts were extracted from 4 distinct musical pieces. The algorithms were evaluated on their precision in matching events in the audio to the score. | ||
| Line 8: | Line 8: | ||
====Team ID==== | ====Team ID==== | ||
| − | '''cont''' = [https://www.music-ir.org/ | + | '''cont''' = [https://www.music-ir.org/mirex/abstracts/2006/SF_cont.pdf Arshia Cont and Diemo Schwarz]<br /> |
| − | '''puckette''' = [https://www.music-ir.org/ | + | '''puckette''' = [https://www.music-ir.org/mirex/abstracts/2006/SF_puckette.pdf Miller Puckette]<br /> |
==Overall Summary Results== | ==Overall Summary Results== | ||
===MIREX 2006 Score Following Summary Results=== | ===MIREX 2006 Score Following Summary Results=== | ||
| − | <csv>sf06_sum.csv</csv> | + | <csv>2006/sf06_sum.csv</csv> |
| − | + | ||
===Results - Cont and Schwarz=== | ===Results - Cont and Schwarz=== | ||
| − | <csv>arshia_evaluation_results.csv</csv> | + | <csv>2006/arshia_evaluation_results.csv</csv> |
===Results - Puckette=== | ===Results - Puckette=== | ||
| − | <csv>mp_evaluation_results.csv</csv> | + | <csv>2006/mp_evaluation_results.csv</csv> |
Latest revision as of 19:33, 13 May 2010
Contents
Introduction
These are the results for the 2006 running of the Score Following task set. For background information about this task set please refer to the 2006:Score Following page.
The aim of the Score Following task is the real-time alignment of incoming music signals to their music score. In this task, 46 recordings and their corresponding MIDI representations of the score were used in the evaluation. These 46 excerpts were extracted from 4 distinct musical pieces. The algorithms were evaluated on their precision in matching events in the audio to the score.
General Legend
Team ID
cont = Arshia Cont and Diemo Schwarz
puckette = Miller Puckette
Overall Summary Results
MIREX 2006 Score Following Summary Results
| Contestant | Total Precision | Piecewise Precision |
|---|---|---|
| cont | 82.90% | 90.06% |
| puckette | 29.75% | 69.74% |
Results - Cont and Schwarz
| Filename | Event Count | #Missed | False Pos | Average Offset | Mean offset | std_offset | ave_latency | missednote | False Positive |
|---|---|---|---|---|---|---|---|---|---|
| mozart_dorabella.001 | 76 | 12 | 0 | 409.7 | 220.2 | 905.4 | 0 | 15.79% | 0.00% |
| mozart_dorabella.002 | 76 | 12 | 0 | 304.5 | 117.3 | 647 | 0 | 15.79% | 0.00% |
| mozart_dorabella.003 | 77 | 2 | 0 | 91.8 | -40.4 | 148 | 0 | 2.60% | 0.00% |
| mozart_dorabella.004 | 147 | 10 | 0 | 203.1 | -0.4 | 444.3 | 0 | 6.80% | 0.00% |
| mozart_k370.030 | 915 | 103 | 0 | 142.3 | -69.2 | 231.4 | 0 | 11.26% | 0.00% |
| mozart_k370.032 | 915 | 53 | 0 | 118.2 | -31 | 324.3 | 0 | 5.79% | 0.00% |
| mozart_k370.034 | 440 | 114 | 0 | 115.1 | 18.7 | 224.7 | 0 | 25.91% | 0.00% |
| mozart_k370.036 | 440 | 115 | 0 | 167.4 | 65.6 | 407.8 | 0 | 26.14% | 0.00% |
| s01 | 88 | 0 | 0 | 71.1 | -59.7 | 69.2 | 0 | 0.00% | 0.00% |
| s03 | 21 | 0 | 0 | 50.3 | -45.2 | 28.7 | 0 | 0.00% | 0.00% |
| s04 | 76 | 0 | 0 | 54.9 | -54.9 | 16.5 | 0 | 0.00% | 0.00% |
| s05 | 23 | 12 | 0 | 375.9 | -375.9 | 690 | 0 | 52.17% | 0.00% |
| s06 | 108 | 0 | 0 | 66.7 | -66.7 | 11.4 | 0 | 0.00% | 0.00% |
| s07 | 45 | 2 | 0 | 108.6 | -108.2 | 64.4 | 0 | 4.44% | 0.00% |
| s08 | 35 | 8 | 0 | 103 | -103 | 34.3 | 0 | 22.86% | 0.00% |
| s09 | 56 | 0 | 0 | 78.1 | -78.1 | 15.6 | 0 | 0.00% | 0.00% |
| s11 | 63 | 4 | 0 | 103.1 | -5 | 257.1 | 0 | 6.35% | 0.00% |
| s12 | 83 | 3 | 1 | 69.6 | -70.5 | 15.6 | 0 | 3.61% | 1.20% |
| s13 | 54 | 0 | 0 | 78.7 | -78.7 | 16.1 | 0 | 0.00% | 0.00% |
| s14 | 16 | 0 | 0 | 91.4 | -91.4 | 23.6 | 0 | 0.00% | 0.00% |
| s15 | 41 | 0 | 0 | 107.7 | -82.2 | 85.1 | 0 | 0.00% | 0.00% |
| s18 | 23 | 1 | 0 | 99.6 | -99.6 | 30.2 | 0 | 4.35% | 0.00% |
| s19 | 42 | 2 | 0 | 62.2 | -62.2 | 27.1 | 0 | 4.76% | 0.00% |
| s20 | 9 | 0 | 0 | 74.5 | -74.5 | 58.2 | 0 | 0.00% | 0.00% |
| t7-s01 | 57 | 1 | 0 | 83.7 | -51.2 | 114.8 | 0 | 1.75% | 0.00% |
| t7-s03 | 90 | 0 | 0 | 92.7 | -78 | 72.8 | 0 | 0.00% | 0.00% |
| t7-s05 | 40 | 0 | 0 | 91.7 | -33.9 | 136 | 0 | 0.00% | 0.00% |
| t7-s06 | 84 | 1 | 0 | 140.6 | -55.3 | 189.2 | 0 | 1.19% | 0.00% |
| t7-s08 | 53 | 0 | 0 | 93.1 | -93.1 | 26.5 | 0 | 0.00% | 0.00% |
| t7-s10 | 25 | 0 | 0 | 79.2 | -70.1 | 41.2 | 0 | 0.00% | 0.00% |
| t7-s12 | 126 | 10 | 0 | 77.4 | -77.4 | 62.6 | 0 | 7.94% | 0.00% |
| t7-s13 | 52 | 0 | 0 | 57.9 | -57.9 | 22.5 | 0 | 0.00% | 0.00% |
| t7-s14 | 37 | 3 | 0 | 515.2 | 365.2 | 1056.2 | 0 | 8.11% | 0.00% |
| t7-s15 | 39 | 0 | 0 | 69.8 | -69.8 | 23.3 | 0 | 0.00% | 0.00% |
| t7-s16 | 98 | 3 | 0 | 147.1 | -87.4 | 181.2 | 0 | 3.06% | 0.00% |
| t7-s17 | 51 | 0 | 0 | 81 | -58.9 | 83.1 | 0 | 0.00% | 0.00% |
| t7-s18 | 90 | 6 | 0 | 70 | -70 | 49.8 | 0 | 6.67% | 0.00% |
| t7-s19 | 93 | 2 | 0 | 138.4 | -123.3 | 156 | 0 | 2.15% | 0.00% |
| t7-s20 | 34 | 0 | 0 | 100.1 | -100.1 | 34.8 | 0 | 0.00% | 0.00% |
| t7-s21 | 92 | 3 | 0 | 114 | -97.4 | 73.4 | 0 | 3.26% | 0.00% |
| t7-s22 | 117 | 8 | 0 | 85.4 | -56.3 | 138.8 | 0 | 6.84% | 0.00% |
| t7-s23 | 20 | 12 | 0 | 1278.9 | -1278.9 | 2420.4 | 0 | 60.00% | 0.00% |
| t7-s25 | 41 | 6 | 0 | 126.2 | -126.2 | 42 | 0 | 14.63% | 0.00% |
| vs1-1ada | 364 | 335 | 197 | 258.5 | 1891.7 | 2070.3 | 0 | 92.03% | 54.12% |
| vs1-4prs | 1595 | 428 | 0 | 1997.1 | 18.1 | 4289.7 | 0 | 26.83% | 0.00% |
| vs2-4alg | 2037 | 286 | 0 | 604.7 | 598.6 | 371.5 | 0 | 14.04% | 0.00% |
Results - Puckette
| Filename | Event Count | #Missed | False Pos | Average Offset | Mean offset | std_offset | ave_latency | missednote | False Positive |
|---|---|---|---|---|---|---|---|---|---|
| mozart_dorabella.001 | 76 | 37 | 3 | 940 | 916.2 | 1689.1 | 20 | 48.68% | 3.95% |
| mozart_dorabella.002 | 76 | 31 | 0 | 1500.1 | 1464.1 | 2035.7 | 20 | 40.79% | 0.00% |
| mozart_dorabella.003 | 77 | 45 | 11 | 1661.3 | 2196.9 | 1697.2 | 20 | 58.44% | 14.29% |
| mozart_dorabella.004 | 147 | 57 | 3 | 687.1 | 627.5 | 1145.3 | 20 | 38.78% | 2.04% |
| mozart_k370.030 | 915 | 890 | 162 | 237.6 | 1777.3 | 998.1 | 20 | 97.27% | 17.70% |
| mozart_k370.032 | 915 | 891 | 159 | 238 | 1814.8 | 1080.7 | 20 | 97.38% | 17.38% |
| mozart_k370.034 | 440 | 93 | 0 | 142.7 | 45.7 | 468.9 | 20 | 21.14% | 0.00% |
| mozart_k370.036 | 440 | 106 | 0 | 123.7 | 29.2 | 370.8 | 20 | 24.09% | 0.00% |
| s01 | 88 | 8 | 0 | 54.1 | -50.7 | 48.7 | 19.9 | 9.09% | 0.00% |
| s03 | 21 | 3 | 0 | 132.5 | 63.8 | 300.4 | 19.6 | 14.29% | 0.00% |
| s04 | 76 | 12 | 0 | 141.3 | 51.8 | 539.5 | 19.9 | 15.79% | 0.00% |
| s05 | 23 | 5 | 0 | 168.7 | -79.7 | 279.1 | 20 | 21.74% | 0.00% |
| s06 | 108 | 12 | 0 | 121.5 | 22.1 | 351.9 | 19.9 | 11.11% | 0.00% |
| s07 | 45 | 17 | 0 | 665.4 | -280.2 | 1184.1 | 20 | 37.78% | 0.00% |
| s08 | 35 | 20 | 0 | 1488.3 | 1447.5 | 1998.5 | 19.5 | 57.14% | 0.00% |
| s09 | 56 | 8 | 1 | 253.9 | 133.8 | 996.4 | 19.8 | 14.29% | 1.79% |
| s11 | 63 | 16 | 0 | 142.1 | 53.4 | 493.5 | 19.8 | 25.40% | 0.00% |
| s12 | 83 | 24 | 1 | 344.3 | 44.6 | 859.1 | 20 | 28.92% | 1.20% |
| s13 | 54 | 12 | 0 | 479.5 | 385 | 1024 | 19.8 | 22.22% | 0.00% |
| s14 | 16 | 5 | 0 | 329 | -153.2 | 727.3 | 19.3 | 31.25% | 0.00% |
| s15 | 41 | 3 | 0 | 157.3 | 2.9 | 368.7 | 19.8 | 7.32% | 0.00% |
| s18 | 23 | 12 | 5 | 282 | -410.2 | 1100.7 | 19.8 | 52.17% | 21.74% |
| s19 | 42 | 12 | 3 | 171.2 | 37 | 635.7 | 19.8 | 28.57% | 7.14% |
| s20 | 9 | 4 | 0 | 174.6 | -174.6 | 207.7 | 20 | 44.44% | 0.00% |
| t7-s01 | 57 | 6 | 0 | 99.1 | -12.2 | 187.1 | 19.8 | 10.53% | 0.00% |
| t7-s03 | 90 | 8 | 0 | 129.5 | -31.2 | 196.2 | 20 | 8.89% | 0.00% |
| t7-s05 | 40 | 6 | 0 | 74.6 | -20.4 | 161.6 | 20 | 15.00% | 0.00% |
| t7-s06 | 84 | 7 | 0 | 78.7 | 5.9 | 176.1 | 19.9 | 8.33% | 0.00% |
| t7-s08 | 53 | 7 | 1 | 135.4 | -5.8 | 268.9 | 20 | 13.21% | 1.89% |
| t7-s10 | 25 | 1 | 0 | 206.7 | 1.3 | 423.8 | 19.9 | 4.00% | 0.00% |
| t7-s12 | 126 | 21 | 0 | 160.3 | 88.1 | 673.8 | 20 | 16.67% | 0.00% |
| t7-s13 | 52 | 9 | 0 | 282.8 | 211.2 | 803.8 | 20 | 17.31% | 0.00% |
| t7-s14 | 37 | 22 | 5 | 1407.4 | 1809.1 | 2239.3 | 19.9 | 59.46% | 13.51% |
| t7-s15 | 39 | 3 | 0 | 45.1 | -45.1 | 21.6 | 20 | 7.69% | 0.00% |
| t7-s16 | 98 | 4 | 0 | 60.6 | -52.3 | 60.8 | 19.9 | 4.08% | 0.00% |
| t7-s17 | 51 | 2 | 0 | 58.4 | -57.8 | 32.1 | 19.8 | 3.92% | 0.00% |
| t7-s18 | 90 | 8 | 0 | 76.2 | -23.1 | 182.6 | 20 | 8.89% | 0.00% |
| t7-s19 | 93 | 5 | 0 | 111.9 | -30 | 207.4 | 20 | 5.38% | 0.00% |
| t7-s20 | 34 | 7 | 0 | 408.4 | 279 | 826.8 | 20 | 20.59% | 0.00% |
| t7-s21 | 92 | 8 | 0 | 80.7 | -36.3 | 114.1 | 20 | 8.70% | 0.00% |
| t7-s22 | 117 | 19 | 1 | 77.9 | -15 | 151 | 20 | 16.24% | 0.85% |
| t7-s23 | 20 | 4 | 0 | 552.1 | 446.2 | 1126.8 | 20 | 20.00% | 0.00% |
| t7-s25 | 41 | 0 | 0 | 71.7 | -68.3 | 34.9 | 20 | 0.00% | 0.00% |
| vs1-1ada | 364 | 358 | 146 | 145 | 3645 | 2856.4 | 20 | 98.35% | 40.11% |
| vs1-4prs | 1595 | 1568 | 550 | 147.7 | 3146.6 | 1717 | 20 | 98.31% | 34.48% |
| vs2-4alg | 2037 | 2000 | 847 | 119.2 | 2848.9 | 1590.1 | 20 | 98.18% | 41.58% |
