Difference between revisions of "Talk:Symbolic Melodic"
From MIREX Wiki
(→Open points) |
(My two cents) |
||
| Line 2: | Line 2: | ||
* Copyright. Any opinions on whether we need to contact RISM? If so, what would be the best way? | * Copyright. Any opinions on whether we need to contact RISM? If so, what would be the best way? | ||
| + | |||
| + | == My two cents == | ||
| + | |||
| + | It is great to have the ground truth set up. Thank you for the painstaking efforts! And I hope more queries can be added in the future. | ||
| + | |||
| + | For the second evaluation, we can try the pool method in TREC: let experts judge on items retreved by all participant teams, items not retrieved by any team are assumed to be irrelevant. It is fair to all participants, but need experts' judgment. | ||
| + | |||
| + | For the first evaluation, I agree we will need to consider those items ignored in building the ground truth. Again, experts' judgment is needed. Can we ask music librarian or music students for help? | ||
| + | |||
| + | Thank you! | ||
Revision as of 20:44, 20 February 2005
Open points
- Copyright. Any opinions on whether we need to contact RISM? If so, what would be the best way?
My two cents
It is great to have the ground truth set up. Thank you for the painstaking efforts! And I hope more queries can be added in the future.
For the second evaluation, we can try the pool method in TREC: let experts judge on items retreved by all participant teams, items not retrieved by any team are assumed to be irrelevant. It is fair to all participants, but need experts' judgment.
For the first evaluation, I agree we will need to consider those items ignored in building the ground truth. Again, experts' judgment is needed. Can we ask music librarian or music students for help?
Thank you!
