2014:Audio Music Similarity and Retrieval Results - Revision history

Yun Hao: /* Reports */

2015-02-05T03:55:19Z

‎Reports

Yun Hao at 04:53, 16 October 2014

2014-10-16T04:53:48Z

Yun Hao: /* Introduction */

2014-10-16T04:31:31Z

‎Introduction

Yun Hao at 04:31, 16 October 2014

2014-10-16T04:31:09Z

Yun Hao: Created page with "== Introduction == These are the results for the 2013 running of the Audio Music Similarity and Retrieval task set. For background information about this task set please refer to..."

2014-01-07T21:32:39Z

Created page with "== Introduction == These are the results for the 2013 running of the Audio Music Similarity and Retrieval task set. For background information about this task set please refer to..."

New page

== Introduction ==
These are the results for the 2013 running of the Audio Music Similarity and Retrieval task set. For background information about this task set please refer to the Audio Music Similarity and Retrieval page.

Each system was given 7000 songs chosen from IMIRSEL's "uspop", "uscrap" and "american" "classical" and "sundry" collections. Each system then returned a 7000x7000 distance matrix. 50 songs were randomly selected from the 10 genre groups (5 per genre) as queries and the first 5 most highly ranked songs out of the 7000 were extracted for each query (after filtering out the query itself, returned results from the same artist were also omitted). Then, for each query, the returned results (candidates) from all participants were grouped and were evaluated by human graders using the Evalutron 6000 grading system. Each individual query/candidate set was evaluated by a single grader. For each query/candidate pair, graders provided two scores. Graders were asked to provide 1 categorical '''BROAD''' score with 3 categories: NS,SS,VS as explained below, and one '''FINE''' score (in the range from 0 to 100). A description and analysis is provided below.

The systems read in 30 second audio clips as their raw data. The same 30 second clips were used in the grading stage.

=== General Legend ===

==== Team ID ====
{| border="1" cellspacing="0" style="text-align: left; width: 800px;"
|- style="background: yellow;"
! width="80" | Sub code
! width="200" | Submission name
! width="80" style="text-align: center;" | Abstract
! width="440" | Contributors
|-
! DM1
| DM1 || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/DM1.pdf PDF] || [http://www.iam.ecs.soton.ac.uk/ Franz de Leon], [http://www.iam.ecs.soton.ac.uk/ Kirk Martinez]
|-
! DM2
| DM2 || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/DM3.pdf PDF] || [http://www.iam.ecs.soton.ac.uk/ Franz de Leon], [http://www.iam.ecs.soton.ac.uk/ Kirk Martinez]
|-
! GKC1
| ILSP Audio Similarity || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/GKC1.pdf PDF] || [http://www.ilsp.gr/ Aggelos Gkiokas], [http://www.ilsp.gr/ Vassilis Katsouros], [http://www.ece.ntua.gr/index.php?option=com_dep&task=profile&id=12&Itemid=62&lang=en George Carayannis]
|-
! GKC2
| ILSP Audio Similarity 2 || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/GKC2.pdf PDF] || [http://www.ilsp.gr/en/infoprojects/meta?view=member&task=show&id=20 Aggelos Gkiokas], [http://www.ilsp.gr/en/research/publications?view=member&id=33&task=show Vassilis Katsouros], [http://www.ilsp.gr/homepages/carayannis_eng.html George Carayannis]
|-
! PS1
| PS09 || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2011/PS1.pdf PDF] || [http://www.ofai.at/~dominik.schnitzer Dominik Schnitzer], [http://www.cp.jku.at/ Tim Pohle]
|-
! RA1
| SimRet2013 || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/RA1.pdf PDF] || [http://www.bsu.by/en/main.aspx Roman Aliyev]
|-
! SS2
| cbmr_sim_2013_submission || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/SS2.pdf PDF] || [http://www.seyerlehner.info Klaus Seyerlehner], [http://www.cp.jku.at Markus Schedl]
|-
! SSPK1
| cbmr_sim_2010_resubmission || style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2012/SSKS2.pdf PDF] || [http://www.seyerlehner.info Klaus Seyerlehner], [http://www.cp.jku.at Markus Schedl], [http://www.cp.jku.at/ Tim Pohle], [http://www.cp.jku.at Peter Knees]
|-
|}

====Broad Categories====
'''NS''' = Not Similar 
'''SS''' = Somewhat Similar 
'''VS''' = Very Similar 

=====Understanding Summary Measures=====
'''Fine''' = Has a range from 0 (failure) to 100 (perfection). 
'''Broad''' = Has a range from 0 (failure) to 2 (perfection) as each query/candidate pair is scored with either NS=0, SS=1 or VS=2. 

==Human Evaluation==
===Overall Summary Results===

<csv p=3>2013/ams/2013_summary_evalutron.csv</csv>

===Friedman's Tests===
====Friedman's Test (FINE Scores)====
The Friedman test was run in MATLAB against the '''Fine''' summary data over the 50 queries. 
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

<csv p=3>2013/ams/2013_evalutron.fine.friedman.tukeyKramerHSD.csv</csv>

[[File:2013_evalutron.fine.friedman.tukeyKramerHSD.png|500px]]

====Friedman's Test (BROAD Scores)====
The Friedman test was run in MATLAB against the '''BROAD''' summary data over the 50 queries. 
Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);

<csv p=3>2013/ams/2013_evalutron.cat.friedman.tukeyKramerHSD.csv</csv>

[[File:2013_evalutron.cat.friedman.tukeyKramerHSD.png|500px]]

===Summary Results by Query===
====FINE Scores====
These are the mean FINE scores per query assigned by Evalutron graders. The FINE scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0 and 100. A perfect score would be 100. Genre labels have been included for reference.

<csv p=1>2013/ams/2013_fine_scores.csv</csv>

====BROAD Scores====
These are the mean BROAD scores per query assigned by Evalutron graders. The BROAD scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0 (not similar) and 2 (very similar). A perfect score would be 2. Genre labels have been included for reference.

<csv p=1>2013/ams/2013_cat_scores.csv</csv>

===Raw Scores===
The raw data derived from the Evalutron 6000 human evaluations are located on the [[2013:Audio Music Similarity and Retrieval Raw Data]] page.

==Metadata and Distance Space Evaluation==
The following reports provide evaluation statistics based on analysis of the distance space and metadata matches and include:
* Neighbourhood clustering by artist, album and genre
* Artist-filtered genre clustering
* How often the triangular inequality holds
* Statistics on 'hubs' (tracks similar to many tracks) and orphans (tracks that are not similar to any other tracks at N results).

=== Reports ===

'''DM1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/DM1/report.txt Franz de Leon, Kirk Martinez] 
'''DM2''' = [https://music-ir.org/mirex/results/2013/ams/statistics/DM2/report.txt Franz de Leon, Kirk Martinez] 
'''GKC1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/GKC1/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis] 
'''GKC2''' = [https://music-ir.org/mirex/results/2013/ams/statistics/GKC2/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis] 
'''PS1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/PS1/report.txt Dominik Schnitzer, Tim Pohle] 
'''RA1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/RA1/report.txt Roman Aliyev] 
'''SS2''' = [https://music-ir.org/mirex/results/2013/ams/statistics/SS2/report.txt Klaus Seyerlehner, Markus Schedl] 
'''SSPK1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/SSPK1/report.txt Klaus Seyerlehner, Markus Schedl, Tim Pohle, Peter Knees]

@@ Line 98: / Line 98: @@
 === Reports ===
-'''FFMD1''' = [https://music-ir.org/mirex/results/2014/ams/statistics/FFMD1/report.txt Peter Foster, György Fazekas, Matthias Mauch, Simon Dixon]<br />
+'''FFMD1''' = [https://music-ir.org/mirex/results/2014/ams/statsReports/FFMD1/report.txt Peter Foster, György Fazekas, Matthias Mauch, Simon Dixon]<br />
-'''GK1''' = [https://music-ir.org/mirex/results/2014/ams/statistics/GK1/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
+'''GK1''' = [https://music-ir.org/mirex/results/2014/ams/statsReports/GK1/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
-'''GKC2''' = [https://music-ir.org/mirex/results/2014/ams/statistics/GKC2/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
+'''GKC2''' = [https://music-ir.org/mirex/results/2014/ams/statsReports/GKC2/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
-'''GKC3''' = [https://music-ir.org/mirex/results/2014/ams/statistics/GKC3/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
+'''GKC3''' = [https://music-ir.org/mirex/results/2014/ams/statsReports/GKC3/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
-'''PS1''' = [https://music-ir.org/mirex/results/2014/ams/statistics/PS1/report.txt Tim Pohle, Dominik Schnitzer]<br />
+'''PS1''' = [https://music-ir.org/mirex/results/2014/ams/statsReports/PS1/report.txt Tim Pohle, Dominik Schnitzer]<br />
-'''SS2''' = [https://music-ir.org/mirex/results/2014/ams/statistics/SS2/report.txt Klaus Seyerlehner, Markus Schedl]<br />
+'''SS2''' = [https://music-ir.org/mirex/results/2014/ams/statsReports/SS2/report.txt Klaus Seyerlehner, Markus Schedl]<br />
-'''SS4''' = [https://music-ir.org/mirex/results/2014/ams/statistics/SS4/report.txt Klaus Seyerlehner, Markus Schedl]<br />
+'''SS4''' = [https://music-ir.org/mirex/results/2014/ams/statsReports/SS4/report.txt Klaus Seyerlehner, Markus Schedl]<br />
-'''SS5''' = [https://music-ir.org/mirex/results/2014/ams/statistics/SS5/report.txt Abhinava Srivastava, Mohit Sinha]<br />
+'''SS5''' = [https://music-ir.org/mirex/results/2014/ams/statsReports/SS5/report.txt Abhinava Srivastava, Mohit Sinha]<br />

@@ Line 19: / Line 19: @@
 	|-
          ! FFMD1
-	| FFMD ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/FFMD1.pdf PDF] || Peter Foster, György Fazekas, Matthias Mauch, Simon Dixon
+	| FFMD ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/FFMD1.pdf PDF] || [http://www.eecs.qmul.ac.uk/~peterf/ Peter Foster], [http://c4dm.eecs.qmul.ac.uk/people/gyorgyf.htm György Fazekas], [http://www.eecs.qmul.ac.uk/people/view/2932/dr-matthias-mauch Matthias Mauch], [http://www.eecs.qmul.ac.uk/~simond/ Simon Dixon]
          |-
          ! GK1

@@ Line 1: / Line 1: @@
 == Introduction ==
-These are the results for the 2013 running of the Audio Music Similarity and Retrieval task set. For background information about this task set please refer to the Audio Music Similarity and Retrieval page.
+These are the results for the 2014 running of the Audio Music Similarity and Retrieval task set. For background information about this task set please refer to the Audio Music Similarity and Retrieval page.
 Each system was given 7000 songs chosen from IMIRSEL's "uspop", "uscrap" and "american" "classical" and "sundry" collections. Each system then returned a 7000x7000 distance matrix. 50 songs were randomly selected from the 10 genre groups (5 per genre) as queries and the first 5 most highly ranked songs out of the 7000 were extracted for each query (after filtering out the query itself, returned results from the same artist were also omitted). Then, for each query, the returned results (candidates) from all participants were grouped and were evaluated by human graders using the Evalutron 6000 grading system. Each individual query/candidate set was evaluated by a single grader. For each query/candidate pair, graders provided two scores. Graders were asked to provide 1 categorical '''BROAD''' score with 3 categories: NS,SS,VS as explained below, and one '''FINE''' score (in the range from 0 to 100). A description and analysis is provided below.
@@ Line 19: / Line 19: @@
 	|-
          ! FFMD1
-	| FFMD ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/FFMD1.pdf PDF] || [Peter Foster, György Fazekas, Matthias Mauch, Simon Dixon]
+	| FFMD ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/FFMD1.pdf PDF] || Peter Foster, György Fazekas, Matthias Mauch, Simon Dixon
          |-
          ! GK1

@@ Line 18: / Line 18: @@
 	! width="440" | Contributors
 	|-
-         ! DM1
+         ! FFMD1
-	| DM1 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/DM1.pdf PDF] || [http://www.iam.ecs.soton.ac.uk/ Franz	de Leon], [http://www.iam.ecs.soton.ac.uk/ Kirk	Martinez]
+	| FFMD ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/FFMD1.pdf PDF] || [Peter Foster, György Fazekas, Matthias Mauch, Simon Dixon]
          |-
-         ! DM2
+         ! GK1
-	| DM2 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/DM3.pdf PDF] || [http://www.iam.ecs.soton.ac.uk/ Franz	de Leon], [http://www.iam.ecs.soton.ac.uk/ Kirk	Martinez]
+	| ILSP AMS ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GK1.pdf PDF] ||  [http://www.ilsp.gr/en/infoprojects/meta?view=member&task=show&id=20 Aggelos Gkiokas], [http://www.ilsp.gr/en/research/publications?view=member&id=33&task=show Vassilis Katsouros], [http://www.ilsp.gr/homepages/carayannis_eng.html George Carayannis]
 	|-
-         ! GKC1
+         ! GKC2
-	| ILSP Audio Similarity ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/GKC1.pdf PDF] || [http://www.ilsp.gr/ Aggelos Gkiokas], [http://www.ilsp.gr/ Vassilis Katsouros], [http://www.ece.ntua.gr/index.php?option=com_dep&task=profile&id=12&Itemid=62&lang=en George Carayannis]
+	| ILSP AMS 2 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GKC2.pdf PDF] ||  [http://www.ilsp.gr/en/infoprojects/meta?view=member&task=show&id=20 Aggelos Gkiokas], [http://www.ilsp.gr/en/research/publications?view=member&id=33&task=show Vassilis Katsouros], [http://www.ilsp.gr/homepages/carayannis_eng.html George Carayannis]
          |-
-         ! GKC2
+         ! GKC3
-	| ILSP Audio Similarity 2 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/GKC2.pdf PDF] || [http://www.ilsp.gr/en/infoprojects/meta?view=member&task=show&id=20 Aggelos Gkiokas], [http://www.ilsp.gr/en/research/publications?view=member&id=33&task=show Vassilis Katsouros], [http://www.ilsp.gr/homepages/carayannis_eng.html George Carayannis]
+	| ILSP AMS 3 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/GKC3.pdf PDF] || [http://www.ilsp.gr/en/infoprojects/meta?view=member&task=show&id=20 Aggelos Gkiokas], [http://www.ilsp.gr/en/research/publications?view=member&id=33&task=show Vassilis Katsouros], [http://www.ilsp.gr/homepages/carayannis_eng.html George Carayannis]
          |-
          ! PS1
 	| PS09 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2011/PS1.pdf PDF] || [http://www.ofai.at/~dominik.schnitzer Dominik Schnitzer], [http://www.cp.jku.at/ Tim Pohle]
          |-
-         ! RA1
+         ! SS2
-	| SimRet2013 ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/RA1.pdf PDF] || [http://www.bsu.by/en/main.aspx Roman Aliyev]
+	| cbmr_sim_2013_resubmission ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/SS2.pdf PDF] || [http://www.seyerlehner.info Klaus Seyerlehner], [http://www.cp.jku.at Markus Schedl]
          |-
-         ! SS2
+         ! SS4
-	| cbmr_sim_2013_submission ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2013/SS2.pdf PDF] || [http://www.seyerlehner.info Klaus Seyerlehner], [http://www.cp.jku.at Markus Schedl]
+	| cbmr_sim_2014_submission ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/SS4.pdf PDF] || [http://www.seyerlehner.info Klaus Seyerlehner], [http://www.cp.jku.at Markus Schedl]
          |-
-         ! SSPK1
+         ! SS5
-	| cbmr_sim_2010_resubmission ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2012/SSKS2.pdf PDF] || [http://www.seyerlehner.info Klaus Seyerlehner], [http://www.cp.jku.at Markus Schedl], [http://www.cp.jku.at/ Tim Pohle], [http://www.cp.jku.at Peter Knees]
+	| AM-ES ||  style="text-align: center;" | [https://www.music-ir.org/mirex/abstracts/2014/SS5.pdf PDF] || Abhinava Srivastava, Mohit Sinha
          |-
 	|}
@@ Line 56: / Line 56: @@
 ===Overall Summary Results===
-<csv p=3>2013/ams/2013_summary_evalutron.csv</csv>
+<csv p=3>2014/ams/2014_summary_evalutron.csv</csv>
 ===Friedman's Tests===
@@ Line 63: / Line 63: @@
 Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
-<csv p=3>2013/ams/2013_evalutron.fine.friedman.tukeyKramerHSD.csv</csv>
+<csv p=3>2014/ams/2014_evalutron.fine.friedman.tukeyKramerHSD.csv</csv>
-[[File:2013_evalutron.fine.friedman.tukeyKramerHSD.png|500px]]
+[[File:2014_evalutron.fine.friedman.tukeyKramerHSD.png|500px]]
 ====Friedman's Test (BROAD Scores)====
@@ Line 71: / Line 71: @@
 Command: [c,m,h,gnames] = multcompare(stats, 'ctype', 'tukey-kramer','estimate', 'friedman', 'alpha', 0.05);
-<csv p=3>2013/ams/2013_evalutron.cat.friedman.tukeyKramerHSD.csv</csv>
+<csv p=3>2014/ams/2014_evalutron.cat.friedman.tukeyKramerHSD.csv</csv>
-[[File:2013_evalutron.cat.friedman.tukeyKramerHSD.png|500px]]
+[[File:2014_evalutron.cat.friedman.tukeyKramerHSD.png|500px]]
 ===Summary Results by Query===
@@ Line 79: / Line 79: @@
 These are the mean FINE scores per query assigned by Evalutron graders. The FINE scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0 and 100. A perfect score would be 100. Genre labels have been included for reference.
-<csv p=1>2013/ams/2013_fine_scores.csv</csv>
+<csv p=1>2014/ams/2014_fine_scores.csv</csv>
 ====BROAD Scores====
 These are the mean BROAD scores per query assigned by Evalutron graders. The BROAD scores for the 5 candidates returned per algorithm, per query, have been averaged. Values are bounded between 0 (not similar) and 2 (very similar). A perfect score would be 2. Genre labels have been included for reference.
-<csv p=1>2013/ams/2013_cat_scores.csv</csv>
+<csv p=1>2014/ams/2014_cat_scores.csv</csv>
 ===Raw Scores===
-The raw data derived from the Evalutron 6000 human evaluations are located on the [[2013:Audio Music Similarity and Retrieval Raw Data]] page.
+The raw data derived from the Evalutron 6000 human evaluations are located on the [[2014:Audio Music Similarity and Retrieval Raw Data]] page.
 ==Metadata and Distance Space Evaluation==
@@ Line 98: / Line 98: @@
 === Reports ===
-'''DM1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/DM1/report.txt Franz de Leon, Kirk Martinez]<br />
+'''FFMD1''' = [https://music-ir.org/mirex/results/2014/ams/statistics/FFMD1/report.txt Peter Foster, György Fazekas, Matthias Mauch, Simon Dixon]<br />
-'''DM2''' = [https://music-ir.org/mirex/results/2013/ams/statistics/DM2/report.txt Franz de Leon, Kirk Martinez]<br />
+'''GK1''' = [https://music-ir.org/mirex/results/2014/ams/statistics/GK1/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
-'''GKC1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/GKC1/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
+'''GKC2''' = [https://music-ir.org/mirex/results/2014/ams/statistics/GKC2/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
-'''GKC2''' = [https://music-ir.org/mirex/results/2013/ams/statistics/GKC2/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
+'''GKC3''' = [https://music-ir.org/mirex/results/2014/ams/statistics/GKC3/report.txt Aggelos Gkiokas, Vassilis Katsouros, George Carayannis]<br />
-'''PS1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/PS1/report.txt Dominik Schnitzer, Tim Pohle]<br />
+'''PS1''' = [https://music-ir.org/mirex/results/2014/ams/statistics/PS1/report.txt Tim Pohle, Dominik Schnitzer]<br />
-'''RA1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/RA1/report.txt Roman Aliyev]<br />
+'''SS2''' = [https://music-ir.org/mirex/results/2014/ams/statistics/SS2/report.txt Klaus Seyerlehner, Markus Schedl]<br />
-'''SS2''' = [https://music-ir.org/mirex/results/2013/ams/statistics/SS2/report.txt Klaus Seyerlehner, Markus Schedl]<br />
+'''SS4''' = [https://music-ir.org/mirex/results/2014/ams/statistics/SS4/report.txt Klaus Seyerlehner, Markus Schedl]<br />
-'''SSPK1''' = [https://music-ir.org/mirex/results/2013/ams/statistics/SSPK1/report.txt Klaus Seyerlehner, Markus Schedl, Tim Pohle, Peter Knees]<br />
+'''SS5''' = [https://music-ir.org/mirex/results/2014/ams/statistics/SS5/report.txt Abhinava Srivastava, Mohit Sinha]<br />