2006:Evalutron6000 Walkthrough - Revision history

IMIRSELBot: Bot: Fixing image links

2010-05-14T03:33:39Z

Bot: Fixing image links

IMIRSELBot: Robot: Automated text replacement (-\[\[Image:\s*([A-Z]) +[[Image:2006_\1)

2010-05-14T03:28:48Z

Robot: Automated text replacement (-\[\[Image:\s*([A-Z]) +[[Image:2006_\1)

Admin at 01:40, 24 August 2006

2006-08-24T01:40:14Z

Admin at 20:19, 22 August 2006

2006-08-22T20:19:48Z

Admin at 20:18, 22 August 2006

2006-08-22T20:18:01Z

Admin at 20:14, 22 August 2006

2006-08-22T20:14:35Z

Admin at 20:06, 22 August 2006

2006-08-22T20:06:40Z

Admin at 19:57, 22 August 2006

2006-08-22T19:57:34Z

CameronJones at 19:45, 22 August 2006

2006-08-22T19:45:59Z

CameronJones at 19:44, 22 August 2006

2006-08-22T19:44:37Z

@@ Line 10: / Line 10: @@
 When first visiting the Evalutron 6000 homepage, you will see a page similar to this (Fig. 1).
-[[Image:2006_Eval6_home.png]]
+[[Image:2006_eval6_home.png]]
 '''Figure 1. Evalutron 6000 start page.'''
@@ Line 20: / Line 20: @@
 If you have questions about your rights as a subject in this research project, you should contact the UIUC IRB office (http://www.irb.uiuc.edu) for more information. The research protocol for this project is IRB# 07066.
-[[Image:2006_Eval6_register.png]]
+[[Image:2006_eval6_register.png]]
 '''Figure 2. Evalutron 6000 start page.'''
@@ Line 26: / Line 26: @@
 After completing the Registration, the system will ask you to log-in with your newly created username and password. After logging-in, you can begin evaluation (Fig. 3). To start the evaluation process, click on the "Start Evaluation" link on the homepage. You will be presented with several options for what media player you want to use. We have invested considerable effort coding various interfaces to be compatible with the maximum number of browsers, platforms, and players. Most users should be able to use the Flash MP3 Player option -- which we've found to work the best. The Windows Media Player and Quicktime options should also work in most modern browsers.
-[[Image:2006_Eval6_start.png]]
+[[Image:2006_eval6_start.png]]
 '''Figure 3. Audio player selection page.'''
@@ Line 32: / Line 32: @@
 Once you have selected a player configuration, you'll see a query evaluation page (Fig. 4). This page consists of two main parts - the query player and candidate players at the top, and the candidate list below. Each song (query and candidates) has been split into three 30 second clips. The first clip is taken from the first 30 seconds of the audio file (0:00 - 0:30). The second clip is the middle 30 seconds of the song (middle-0:15 - middle+0:15). The third clip is the last 30 seconds of the song (end-0:30 - end). Clicking on these buttons will load the song into the player and begin playing. Depending on the player you are using and your browser's configuration, you should be able to seek through the 30 second clip and replay it using the player's play controls. We recommend you listen to the entire query at least once before evaluating any candidate files.
-[[Image:2006_Eval6_eval_page.png]]
+[[Image:2006_eval6_eval_page.png]]
 '''Figure 4. Sample evaluation page.'''
@@ Line 40: / Line 40: @@
 The procedure for listening to a candidate is the same as listening to a query -- click the 0:30-clip buttons to load the clip into the player and listen to it (Fig. 5). Once you have a feeling for whether or not the candidate is similar to the query, click the "Similar" or "Not Similar" radio buttons to the right of the candidate. Note: you can continue to replay the query during candidate playback.
-[[Image:2006_Eval6_cand_detail.png]]
+[[Image:2006_eval6_cand_detail.png]]
 '''Figure 5. Close up image of candidate manipulation buttons.'''
@@ Line 46: / Line 46: @@
 Depending on how you graded the candidate, you should see the candidate box react to your grade, indicating that the vote has been logged in the database (Fig. 6). If you indicated the song was "Similar" the box will turn green and state "SAVED".
-[[Image:2006_Eval6_sim_saved.png]]
+[[Image:2006_eval6_sim_saved.png]]
 '''Figure 6. Close up image of candidate manipulation buttons with "Similar" selection "SAVED".'''
@@ Line 52: / Line 52: @@
 If you indicated the song was "Not Similar" the box will turn red and state "SAVED" (Fig. 7).
-[[Image:2006_Eval6_notsim_saved.png]]
+[[Image:2006_eval6_notsim_saved.png]]
 '''Figure 7. Close up image of candidate manipulation buttons with "Not Similar" selection "SAVED".'''
@@ Line 60: / Line 60: @@
 When you have completed evaluating all of the candidates for a single query, you can click on the "Next Random Query" button at the bottom of the page (Fig. 8). This button will load a new query and list of candidates for you to evaluate.
-[[Image:2006_Eval6_next_button.png]]
+[[Image:2006_eval6_next_button.png]]
 '''Figure 8. Close up image of "Next Random Query" button which generates next set of candidates for evaluation. '''
@@ Line 66: / Line 66: @@
 You will see a list of all of the queries you have evaluated (or are evaluating) at the bottom of the page. You can return to any query by clicking on the button for that query here (Fig. 9). You can re-evaluate any candidate for any query at any time, up to the closing of the evaluation system.
-[[Image:2006_Eval6_queries_detail.png]]
+[[Image:2006_eval6_queries_detail.png]]
 '''Figure 9. Close up image of Query List buttons which allow for revisiting of completed query/candidate sets. '''

@@ Line 10: / Line 10: @@
 When first visiting the Evalutron 6000 homepage, you will see a page similar to this (Fig. 1).
-[[Image:Eval6_home.png]]
+[[Image:2006_Eval6_home.png]]
 '''Figure 1. Evalutron 6000 start page.'''
@@ Line 20: / Line 20: @@
 If you have questions about your rights as a subject in this research project, you should contact the UIUC IRB office (http://www.irb.uiuc.edu) for more information. The research protocol for this project is IRB# 07066.
-[[Image:Eval6_register.png]]
+[[Image:2006_Eval6_register.png]]
 '''Figure 2. Evalutron 6000 start page.'''
@@ Line 26: / Line 26: @@
 After completing the Registration, the system will ask you to log-in with your newly created username and password. After logging-in, you can begin evaluation (Fig. 3). To start the evaluation process, click on the "Start Evaluation" link on the homepage. You will be presented with several options for what media player you want to use. We have invested considerable effort coding various interfaces to be compatible with the maximum number of browsers, platforms, and players. Most users should be able to use the Flash MP3 Player option -- which we've found to work the best. The Windows Media Player and Quicktime options should also work in most modern browsers.
-[[Image:Eval6_start.png]]
+[[Image:2006_Eval6_start.png]]
 '''Figure 3. Audio player selection page.'''
@@ Line 32: / Line 32: @@
 Once you have selected a player configuration, you'll see a query evaluation page (Fig. 4). This page consists of two main parts - the query player and candidate players at the top, and the candidate list below. Each song (query and candidates) has been split into three 30 second clips. The first clip is taken from the first 30 seconds of the audio file (0:00 - 0:30). The second clip is the middle 30 seconds of the song (middle-0:15 - middle+0:15). The third clip is the last 30 seconds of the song (end-0:30 - end). Clicking on these buttons will load the song into the player and begin playing. Depending on the player you are using and your browser's configuration, you should be able to seek through the 30 second clip and replay it using the player's play controls. We recommend you listen to the entire query at least once before evaluating any candidate files.
-[[Image:Eval6_eval_page.png]]
+[[Image:2006_Eval6_eval_page.png]]
 '''Figure 4. Sample evaluation page.'''
@@ Line 40: / Line 40: @@
 The procedure for listening to a candidate is the same as listening to a query -- click the 0:30-clip buttons to load the clip into the player and listen to it (Fig. 5). Once you have a feeling for whether or not the candidate is similar to the query, click the "Similar" or "Not Similar" radio buttons to the right of the candidate. Note: you can continue to replay the query during candidate playback.
-[[Image:Eval6_cand_detail.png]]
+[[Image:2006_Eval6_cand_detail.png]]
 '''Figure 5. Close up image of candidate manipulation buttons.'''
@@ Line 46: / Line 46: @@
 Depending on how you graded the candidate, you should see the candidate box react to your grade, indicating that the vote has been logged in the database (Fig. 6). If you indicated the song was "Similar" the box will turn green and state "SAVED".
-[[Image:Eval6_sim_saved.png]]
+[[Image:2006_Eval6_sim_saved.png]]
 '''Figure 6. Close up image of candidate manipulation buttons with "Similar" selection "SAVED".'''
@@ Line 52: / Line 52: @@
 If you indicated the song was "Not Similar" the box will turn red and state "SAVED" (Fig. 7).
-[[Image:Eval6_notsim_saved.png]]
+[[Image:2006_Eval6_notsim_saved.png]]
 '''Figure 7. Close up image of candidate manipulation buttons with "Not Similar" selection "SAVED".'''
@@ Line 60: / Line 60: @@
 When you have completed evaluating all of the candidates for a single query, you can click on the "Next Random Query" button at the bottom of the page (Fig. 8). This button will load a new query and list of candidates for you to evaluate.
-[[Image:Eval6_next_button.png]]
+[[Image:2006_Eval6_next_button.png]]
 '''Figure 8. Close up image of "Next Random Query" button which generates next set of candidates for evaluation. '''
@@ Line 66: / Line 66: @@
 You will see a list of all of the queries you have evaluated (or are evaluating) at the bottom of the page. You can return to any query by clicking on the button for that query here (Fig. 9). You can re-evaluate any candidate for any query at any time, up to the closing of the evaluation system.
-[[Image:Eval6_queries_detail.png]]
+[[Image:2006_Eval6_queries_detail.png]]
 '''Figure 9. Close up image of Query List buttons which allow for revisiting of completed query/candidate sets. '''

← Older revision		Revision as of 01:40, 24 August 2006
Line 1:		Line 1:
		+	==UPDATE 23 August 2006==
		+	As of 23 August there is good chance the some of the finer details of the Evalutron 6000 will change in response to community input. One probable change is that the evaluation choices will expand beyond the binary to a tenary scheme for some of the evaluation. Will modify the page once confirmed. Cheers, J. Stephen Downie
		+
		+	==Welcome to the Evalutron 6000==
		+
	In order to use the Evalutron 6000 you will need to be using a modern web browser (e.g., Firefox, Internet Explorer, Safari, Mozilla, etc) that supports JavaScript (ECMAScript) and Cookies. Evalutron has been tested on Windows XP, MacOS X, and RedHat Linux. If you are using a different platform and having trouble, please try accessing Evalutron 6000 from another machine. If you are still having difficulty, contact		In order to use the Evalutron 6000 you will need to be using a modern web browser (e.g., Firefox, Internet Explorer, Safari, Mozilla, etc) that supports JavaScript (ECMAScript) and Cookies. Evalutron has been tested on Windows XP, MacOS X, and RedHat Linux. If you are using a different platform and having trouble, please try accessing Evalutron 6000 from another machine. If you are still having difficulty, contact

← Older revision		Revision as of 20:19, 22 August 2006
Line 63:		Line 63:
	[[Image:Eval6_queries_detail.png]]		[[Image:Eval6_queries_detail.png]]

−	'''Figure 9. Close up image of Query List buttons which allow for revisiting of completed query/candidate sets '''	+	'''Figure 9. Close up image of Query List buttons which allow for revisiting of completed query/candidate sets. '''

@@ Line 3: / Line 3: @@
 mrx-com09@lists.lis.uiuc.edu.
-When first visiting the Evalutron 6000 homepage, you will see a page similar to this.
+When first visiting the Evalutron 6000 homepage, you will see a page similar to this (Fig. 1).
 [[Image:Eval6_home.png]]
@@ Line 11: / Line 11: @@
 First you must register a new account. Click on the "Register" link on the left side of the page to create an account.
-The registration page is fairly straightforward. Required fields are marked in blue with an asterisks. You can create any username and password you wish. Passwords must be at least 6 characters long and are case-sensitive. Before completing the registration, you must read and agree to the terms of the Informed Consent document. The evaluation, because it is using human judgements of similarity, is considered a human-subjects research project and the Evaluatron is basically a survey instrument. To indicate your consent to participate in the evaluation, check the "I Agree" checkbox below the informed consent document.
+The registration page is fairly straightforward (Fig. 2). Required fields are marked in blue with an asterisks. You can create any username and password you wish. Passwords must be at least 6 characters long and are case-sensitive. Before completing the registration, you must read and agree to the terms of the Informed Consent document. The evaluation, because it is using human judgements of similarity, is considered a human-subjects research project and the Evaluatron is basically a survey instrument. To indicate your consent to participate in the evaluation, check the "I Agree" checkbox below the informed consent document.
 If you have questions about your rights as a subject in this research project, you should contact the UIUC IRB office (http://www.irb.uiuc.edu) for more information. The research protocol for this project is IRB# 07066.
@@ Line 19: / Line 19: @@
 '''Figure 2. Evalutron 6000 start page.'''
-After completing the Registration, the system will ask you to log-in with your newly created username and password. After logging-in, you can begin evaluation. To start the evaluation process, click on the "Start Evaluation" link on the homepage. You will be presented with several options for what media player you want to use. We have invested considerable effort coding various interfaces to be compatible with the maximum number of browsers, platforms, and players. Most users should be able to use the Flash MP3 Player option -- which we've found to work the best. The Windows Media Player and Quicktime options should also work in most modern browsers.
+After completing the Registration, the system will ask you to log-in with your newly created username and password. After logging-in, you can begin evaluation (Fig. 3). To start the evaluation process, click on the "Start Evaluation" link on the homepage. You will be presented with several options for what media player you want to use. We have invested considerable effort coding various interfaces to be compatible with the maximum number of browsers, platforms, and players. Most users should be able to use the Flash MP3 Player option -- which we've found to work the best. The Windows Media Player and Quicktime options should also work in most modern browsers.
 [[Image:Eval6_start.png]]
@@ Line 39: / Line 39: @@
 '''Figure 5. Close up image of candidate manipulation buttons.'''
-Depending on how you graded the candidate, you should see the candidate box react to your grade, indicating that the vote has been logged in the database. If you indicated the song was "Similar" the box will turn green and state "SAVED".
+Depending on how you graded the candidate, you should see the candidate box react to your grade, indicating that the vote has been logged in the database (Fig. 6). If you indicated the song was "Similar" the box will turn green and state "SAVED".
 [[Image:Eval6_sim_saved.png]]
@@ Line 45: / Line 45: @@
 '''Figure 6. Close up image of candidate manipulation buttons with "Similar" selection "SAVED".'''
-If you indicated the song was "Not Similar" the box will turn red and state "SAVED".
+If you indicated the song was "Not Similar" the box will turn red and state "SAVED" (Fig. 7).
 [[Image:Eval6_notsim_saved.png]]
@@ Line 53: / Line 53: @@
 You can always change your evaluation for any candidate by toggling the radio buttons. The system should reflect any changes with an appropriate color change. Once an evaluation has been made, however, it cannot be retracted, only reversed. (i.e., you cannot "unvote").
+When you have completed evaluating all of the candidates for a single query, you can click on the "Next Random Query" button at the bottom of the page (Fig. 8). This button will load a new query and list of candidates for you to evaluate.
 [[Image:Eval6_next_button.png]]
@@ Line 60: / Line 59: @@
 '''Figure 8. Close up image of "Next Random Query" button which generates next set of candidates for evaluation. '''
-You will see a list of all of the queries you have evaluated (or are evaluating) at the bottom of the page. You can return to any query by clicking on the button for that query here. You can re-evaluate any candidate for any query at any time, up to the closing of the evaluation system.
+You will see a list of all of the queries you have evaluated (or are evaluating) at the bottom of the page. You can return to any query by clicking on the button for that query here (Fig. 9). You can re-evaluate any candidate for any query at any time, up to the closing of the evaluation system.
 [[Image:Eval6_queries_detail.png]]
-'''Figure 8. Close up image of Query List buttons which allow for revisiting of completed query/candidate sets '''
+'''Figure 9. Close up image of Query List buttons which allow for revisiting of completed query/candidate sets '''

@@ Line 35: / Line 35: @@
 You can always change your evaluation for any candidate by toggling the radio buttons. The system should reflect any changes with an appropriate color change. Once an evaluation has been made, however, it cannot be retracted, only reversed. (i.e., you cannot "unvote").
-Please note that the list of candidates scrolls within the page, there are more candidates that may be immediately visible on the page. Please scroll to the bottom of the candidate list to make sure you've evaluated each song.
+Please note that the list of candidates scrolls within the page, there are more candidates than may be immediately visible on the page. Please scroll to the bottom of the candidate list to make sure you've evaluated each song.
 When you have completed evaluating all of the candidates for a single query, you can click on the "Next Random Query" button at the bottom of the page. This button will load a new query and list of candidates for you to evaluate.

@@ Line 17: / Line 17: @@
 [[Image:Eval6_start.png]]
-Once you have selected a player configuration, you'll see a query evaluation page. This page consists of two main parts - the query player and candidate players at the top, and the candidate list below. Each song (query and candidates) has been split into three 30 second clips. The first clip is taken from the first 30 seconds of the audio file (0:00 - 0:30). The second clip is the middle 30 seconds of the song (middle-0:15 - middle+0:15). The third clip is the last 30 seconds of the song (end-0:30 - end). Clicking on these buttons will load the song into the player and begin playing. Depending on the player you are using and your browser's configuration, you should be able to seek through the 30 second clip and replay it using the players play controls. We recommend you listen to the entire query at least once before evaluating any candidate files.
+Once you have selected a player configuration, you'll see a query evaluation page. This page consists of two main parts - the query player and candidate players at the top, and the candidate list below. Each song (query and candidates) has been split into three 30 second clips. The first clip is taken from the first 30 seconds of the audio file (0:00 - 0:30). The second clip is the middle 30 seconds of the song (middle-0:15 - middle+0:15). The third clip is the last 30 seconds of the song (end-0:30 - end). Clicking on these buttons will load the song into the player and begin playing. Depending on the player you are using and your browser's configuration, you should be able to seek through the 30 second clip and replay it using the player's play controls. We recommend you listen to the entire query at least once before evaluating any candidate files.
 [[Image:Eval6_eval_page.png]]
-The procedure for listening to a candidate is the same as listening to a query -- click the 30-clip buttons to load the clip into the player and listen to it. Once you have a feeling for whether or not the candidate is similar to the query, click the "Similar" or "Not Similar" radio buttons to the right of the candidate. Note: you can continue to replay the query during candidate playback.
+The procedure for listening to a candidate is the same as listening to a query -- click the 0:30-clip buttons to load the clip into the player and listen to it. Once you have a feeling for whether or not the candidate is similar to the query, click the "Similar" or "Not Similar" radio buttons to the right of the candidate. Note: you can continue to replay the query during candidate playback.
 [[Image:Eval6_cand_detail.png]]