2024:Audio Chord Estimation - Revision history

Junyan at 11:53, 4 October 2024

2024-10-04T11:53:37Z

Junyan at 17:26, 15 September 2024

2024-09-15T17:26:45Z

Junyan at 09:39, 15 September 2024

2024-09-15T09:39:55Z

Junyan at 13:47, 10 September 2024

2024-09-10T13:47:03Z

Junyan at 07:17, 25 August 2024

2024-08-25T07:17:30Z

Junyan at 06:33, 25 August 2024

2024-08-25T06:33:23Z

Show changes

Martinhartmann: Created page with "2024:Audio Chord Estimation"

2024-07-30T17:22:49Z

Created page with "2024:Audio Chord Estimation"

New page

2024:Audio Chord Estimation

← Older revision		Revision as of 11:53, 4 October 2024
Line 192:		Line 192:

	Time limit: within 5 times the total duration of the test set.		Time limit: within 5 times the total duration of the test set.
		+
		+	= Contact =
		+
		+	If you have any question, please send an email to <TC: [mailto:jj2731@nyu.edu Junyan Jiang]>.

	= Bibliography =		= Bibliography =

@@ Line 8: / Line 8: @@
 * Submission format: docker image is required. See the submission format section.
-* An online leaderboard: there will be an online leaderboard for the evaluation result on a small validation set. Details will follow when the submission site opens.
+* <del>An online leaderboard: there will be an online leaderboard for the evaluation result on a small validation set. Details will follow when the submission site opens.</del> Postponed to next year
 * Potential new test datasets: we are discussing possibilities with dataset holders to use external proprietary datasets for validation and test sets. The dataset will be private. Details about the dataset's information (number of songs, genre distribution, etc) will follow shortly.

@@ Line 146: / Line 146: @@
 Submissions using machine learning models must also submit their trained models. Training on the evaluation server is no longer supported starting from this year. We will execute the following commands for testing:
-<pre>prepare.sh
+<pre>
-doChordID.sh &quot;/path/to/input1.wav&quot; &quot;/path/to/output1.wav.txt&quot;
+your_program prepare
-doChordID.sh &quot;/path/to/input2.wav&quot; &quot;/path/to/output2.wav.txt&quot;
+your_program do_chord_identification &quot;/path/to/input1.wav&quot; &quot;/path/to/output1.wav.txt&quot;
-...</pre>
+your_program do_chord_identification &quot;/path/to/input2.wav&quot; &quot;/path/to/output2.wav.txt&quot;
-In the results directory, there should be one file for each testfile with same name as the test file + <code>.txt</code>. Programs can use their working directory if they need to keep temporary cache files or internal debugging info. Standard output and standard error will be logged.
+In the results directory, there should be one file for each testfile with same name as the test file + <code>.txt</code>. Programs can use the folder  <code>/app/temp</code> if they need to keep temporary cache files or to download pretrained models. Standard output and standard error will be logged.
-No internet access is allowed during the inference stage (<code>doChordID.sh</code>). Please contact us if your model requires internet access (e.g., model API call) during inference.
+No internet access is allowed during the inference stage (<code>do_chord_identification</code>). Please contact us if your model requires internet access (e.g., model API call) during inference.
 == Packaging submissions ==
 * Every submission must be packed into a docker image
@@ Line 161: / Line 165: @@
 Accepted submission form:
-* Link to public or private Github repository
-* Link to public or private docker hub
+* Docker Hub: You can create a free account at [https://hub.docker.com/ Docker Hub] and upload your docker image there.
-* Shared google drive links
+* Github Container Registry: If you are using Github, you can use the Github Container Registry to upload your docker image.
-* If the repository is private, an access token is also required
+* Google Drive: You can upload your docker image to Google Drive and share the link with the task captain.
 = Time and Hardware limits =

@@ Line 13: / Line 13: @@
 = Data =
-Two datasets are used to evaluate chord transcription accuracy.
+Starting from this year, we do not restrict the training set for machine learning-based systems. However, each submission needs to state clearly the training set they used in their extended abstract. We recommend the following datasets for training:
 ; Isophonics
 : The collected Beatles, Queen, and Zweieck datasets from the Centre for Digital Music at Queen Mary, University of London (http://www.isophonics.net/), as used for Audio Chord Estimation in MIREX for many years. Available from http://www.isophonics.net/. See also Matthias Mauch’s dissertation (2010) and Harte et al.’s introductory paper (2005).
-; Billboard
+; Billboard 2012
 : An abridged version of the ''Billboard'' dataset from McGill University, including a representative sample of American popular music from the 1950s through the 1990s. Available from http://billboard.music.mcgill.ca. See also Ashley Burgoyne’s dissertation (2012) and Burgoyne et al.’s introductory paper (2011). Parsing tools for the data are available from http://hackage.haskell.org/package/billboard-parser/ and documented by De Haas and Burgoyne (2012).
 == Training and Testing ==
 The ground-truth files contain one line per unique chord, in the form <code>{start_time end_time chord}</code>, e.g.,

@@ Line 3: / Line 3: @@
 This task requires participants to extract or transcribe a sequence of chords from an audio music recording. For many applications in music information retrieval, extracting the harmonic structure of an audio track is very desirable, for example for segmenting pieces into characteristic segments, for finding similar pieces, or for semantic analysis of music. The extraction of the harmonic structure requires the estimation of a sequence of chords that is as precise as possible. This includes the full characterisation of chords – root, quality, and bass note – as well as their chronological order, including specific onset times and durations. Audio chord estimation has a long history in MIREX, and readers interested in this history, especially with respect to evaluation methodology, should review the work of Christopher Harte (2010), Pauwels and Peeters (2013), and the [https://www.music-ir.org/mirex/wiki/The_Utrecht_Agreement_on_Chord_Evaluation “Utrecht Agreement”] on evaluation metrics. For python evaluation code, please refer to [https://craffel.github.io/mir_eval/#module-mir_eval.chord “mir_eval”].
-= What's new =
+== What's New ==
 Compared to previous years: