Data and setup

source("./keytime-setup.R")

The data may be downloaded from the project’s page: http://ns.inria.fr/loki/WordSuggestions/.

Runs

This table contains one row per participant. The different columns are as described below:

participant

Participant Identifier.

accuracy

The word suggestion accuracy condition.

keytime

The keytime accuracy condition, i.e. how long keys have to be pressed before they take effect.

suggestions_type

The type of word suggestions used : inline or bar.

start_date

The start date and time of the run in ISO 8601 format.

end_date

The end date and time of the run in ISO 8601 format.

duration

The duration of the run in seconds.

age

The self-reported age of the participant.

gender

The self-reported gender of the participant.

suggestions_use_frequency_desktop

Answer for “In the last 24 hours, how many word suggestions have you used when typing on a desktop computer?”.

suggestions_use_frequency_mobile

Answer for “In the last 24 hours, how many word suggestions have you used when typing on a mobile device?”.

controls_satisfactory

Agreement for “The controls (keyboard and word suggestions) are satisfactory for the completion of the task.” This was provided for each condition.

suggestion_accuracy

Agreement for “The word suggestions are accurate.” This was provided for each condition.

keyboard_use_efficiency

Agreement for “The use of the keyboard is efficient in this task.” This was provided for each condition.

suggestion_distraction

Agreement for “The word suggestions are distracting.” This was provided for each condition.

mental_demand

NASA-TLX mental demand scale. This was provided for each condition.

physical_demand

NASA-TLX physical demand scale. This was provided for each condition.

temporal_demand

NASA-TLX temporal demand scale. This was provided for each condition.

performance

NASA-TLX performance scale. This was provided for each condition.

effort

NASA-TLX performance scale. This was provided for each condition.

frustration

NASA-TLX effort scale. This was provided for each condition.

total_suggestions

The number of suggestions showed to the participant. This is always 1.

total_blurred_measured_trials

Register the number of trials during which the page was unfocused, which is an indication that the participant may be distracted.

total_blurred_measured_trials

Register the number of typing trials during which the page was unfocused, which is an indication that the participant may be distracted.

is_run_completed

TRUE if the experiment was completed.

is_run_measured

One participant was removed because they misunderstood they though they had to use all suggestions. Another participant was removed because they participated more than once.

is_participant_trusted

TRUE if the participant passed the attention test. If not, the participant is excluded from subjective and demographic analysis.

run_rejection_reason

The reason why the participant was fully removed from the analysis. This is only set if is_run_measured is FALSE.

runs <- read_keytime_runs(measured_only = FALSE)
measured_runs <- runs |> filter(is_run_measured)
runs

Trials

This table contains data logs from all participant trials.

participant

Participant Identifier.

accuracy

The word suggestion accuracy condition.

keytime

The device used for the run.

suggestions_type

The type of word suggestions used: inline or bar.

trial_number

The number of the trial.

trial_id

The identifier of the trial.

is_practice

If the trial was part of a practice block.

phrase

The phrase participant had to type during the trial.

total_chars

The number of character in the phrase to type.

theoretical_sks

The maximum number of keystrokes that could be saved using word suggestions during the trial.

start_date

The start date and time of the trial in ISO 8601 format.

end_date

The end date and time of the trial in ISO 8601 format.

duration

The duration of the trial in seconds.

actual_sks

The number of keystrokes that were saved using word suggestions. This may be negative if incorrect suggestions were used.

total_key_strokes

The number of key that were pressed to complete the trial.

total_suggestion_used

The number of suggestions that were used during the trial.

total_suggestion_errors

The number of incorrect suggestions that were used during the trial.

total_removed_manual_chars

The number of characters that were manually entered then deleted by participant during the trial.

total_removed_suggestion_chars

The number of characters that were inserted from a suggestion then deleted by participant during the trial.

total_final_manual_chars

The number of characters from the final participant input that were manually inserted.

total_added_chars

The number of characters that were inserted during the trial either from suggestion or manual input (some may have been removed later).

does_trial_have_errors

TRUE if incorrect characters were inserted at some point during the trial (they had to be removed later to validate the trial).

phrase_completion_start

The date and time where the first character was inserted during the trial (ISO 8601 format).

phrase_completion_end

The date and time where the phrase was fully completed and correct for the first time (ISO 8601 format). Note that this is before the trial was validated.

phrase_completion_duration

The time it took for the participant to complete the phrase in seconds.

total_blur_events

The number of times the experiment web page was unfocused during the trial.

was_trial_blurred

TRUE if total_blur_events > 0

theoretical_key_saving

The maximum key stroke saving of the trial. This should be close from accuracy.

actual_key_saving_no_editing

The key stroke saving excluding errors.

actual_key_saving

The key stroke saving.

cps

Entry speed in characters per second (i.e. (total_chars - 1) / trial_duration).

is_run_measured

If the run including this trial was included in our statistical analysis.

is_trial_measured

If this trial was included in our statistical analysis. See below for reasons to exclude a trials from our analysis.

trial_rejection_reason

The reason for excluding the trial from the analysis. This is only provided for trials that were excluded.

trials <- read_keytime_trials(measured_only = FALSE)
measured_trials <- trials |> filter(is_trial_measured)
trials

Events

This large table contains all event logs recorded during our experiment. It includes the following columns:

participant

The participant identifier.

accuracy

The accuracy condition.

keytime

The keytime condition.

suggestions_type

The type of word suggestions used: inline or bar.

trial_id

The trial identifier.

event_number

The number of the event in the trial.

type

The type of the event. INIT: trial initialization, UPDATE_SUGGESTIONS: update of the suggestions, INPUT_CHAR: manual insertion of a character by the participant at the end of their input, DELETE_CHAR: removal of the last input character by the participant, INPUT_SUGGESTION: insertion of a suggestion.

input

The content of participant’s input after the event.

is_input_correct

If there is no errors in the input after the event.

is_error

If the event results from an error, such as choosing an incorrect suggestion or entering an incorrect character.

added_input

The characters added to the input as a result of the event.

removed_input

The characters removed from the input as a result of the event.

total_added_chars

The number of characters added as result of the event (may be negative).

remaining_key_strokes

The number of character left to complete the phrase.

is_target_completed

If the phrase is completed.

time

The date and time of the event (ISO 8601 format).

diff_time

Duration between this event and the previous one.

scheduled_action

The action that was scheduled for this event due to the keytime condition.

is_run_measured

If the run including this event was included in our statistical analysis.

is_trial_measured

If the trial including this event was included in our statistical analysis.

is_event_measured

If this event was included in our statistical analysis.

event_rejection_reason

The reason for excluding the event from the analysis. This is only provided for events that were excluded.

suggestion

The suggestion that was inserted as a result of the event.

target_word

The next word to type or the word currently being typed.

target_word_number

The number of the target word in the phrase.

Monitoring

Total measured runs

measured_runs |> count(suggestions_type)

Total measured trials

measured_trials |> count(suggestions_type)

Blurred Trials

The following trials were removed because the participant was interrupted (the page was unfocused).

trials |> filter(is_run_measured & !is_practice) |>
  count(suggestions_type, was_trial_blurred) |>
  add_count(suggestions_type, wt = n, name="total")

Rejected participants

In addition, the following participant were fully removed from our analysis because they unfocused the page for more than 2 trials.

runs |>
  filter(is_run_completed) |>
  count(suggestions_type, run_rejection_reason)

Untrusted participants

Finally, the following participants were removed from the demographics and subjective analysis because they did not pass the attention test during the questionnaire (however their objective data was kept).

runs |> filter(is_run_measured) |> count(suggestions_type, is_participant_trusted) |>
  add_count(suggestions_type, wt = n, name="total")
LS0tCnRpdGxlOiAiV29yZC1TdWdnZXN0aW9uczogS2V5dGltZSBFeHBlcmltZW50cyIKb3V0cHV0OgogIGh0bWxfbm90ZWJvb2s6CiAgICB0b2M6IHllcwogICAgdG9jX2Zsb2F0OiB5ZXMKICAgIHRoZW1lOiBsdW1lbgogICAgaGlnaGxpZ2h0OiBkZWZhdWx0CiAgICBjb2RlX2ZvbGRpbmc6IGhpZGUKICBodG1sX2RvY3VtZW50OgogICAgdG9jOiB5ZXMKICAgIHRvY19mbG9hdDogeWVzCiAgICB0aGVtZTogbHVtZW4KICAgIGhpZ2hsaWdodDogZGVmYXVsdAogICAgY29kZV9mb2xkaW5nOiBoaWRlCiAgICBkZXY6IHN2Z2xpdGUKLS0tCgojIERhdGEgYW5kIHNldHVwCgpgYGB7ciBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQpzb3VyY2UoIi4va2V5dGltZS1zZXR1cC5SIikKYGBgCgpUaGUgZGF0YSBtYXkgYmUgZG93bmxvYWRlZCBmcm9tIHRoZSBwcm9qZWN0J3MgcGFnZTogPGh0dHA6Ly9ucy5pbnJpYS5mci9sb2tpL1dvcmRTdWdnZXN0aW9ucy8+LgoKIyMgUnVucwoKVGhpcyB0YWJsZSBjb250YWlucyBvbmUgcm93IHBlciBwYXJ0aWNpcGFudC4gVGhlIGRpZmZlcmVudCBjb2x1bW5zIGFyZSBhcyBkZXNjcmliZWQgYmVsb3c6CgpwYXJ0aWNpcGFudAoKOiAgIFBhcnRpY2lwYW50IElkZW50aWZpZXIuCgphY2N1cmFjeQoKOiAgIFRoZSB3b3JkIHN1Z2dlc3Rpb24gYWNjdXJhY3kgY29uZGl0aW9uLgoKa2V5dGltZQoKOiAgIFRoZSBrZXl0aW1lIGFjY3VyYWN5IGNvbmRpdGlvbiwgaS5lLiBob3cgbG9uZyBrZXlzIGhhdmUgdG8gYmUgcHJlc3NlZCBiZWZvcmUgdGhleSB0YWtlIGVmZmVjdC4KCnN1Z2dlc3Rpb25zX3R5cGUKCjogICBUaGUgdHlwZSBvZiB3b3JkIHN1Z2dlc3Rpb25zIHVzZWQgOiBpbmxpbmUgb3IgYmFyLgoKc3RhcnRfZGF0ZQoKOiAgIFRoZSBzdGFydCBkYXRlIGFuZCB0aW1lIG9mIHRoZSBydW4gaW4gW0lTTyA4NjAxXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9JU09fODYwMSkgZm9ybWF0LgoKZW5kX2RhdGUKCjogICBUaGUgZW5kIGRhdGUgYW5kIHRpbWUgb2YgdGhlIHJ1biBpbiBbSVNPIDg2MDFdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL0lTT184NjAxKSBmb3JtYXQuCgpkdXJhdGlvbgoKOiAgIFRoZSBkdXJhdGlvbiBvZiB0aGUgcnVuIGluIHNlY29uZHMuCgphZ2UKCjogICBUaGUgc2VsZi1yZXBvcnRlZCBhZ2Ugb2YgdGhlIHBhcnRpY2lwYW50LgoKZ2VuZGVyCgo6ICAgVGhlIHNlbGYtcmVwb3J0ZWQgZ2VuZGVyIG9mIHRoZSBwYXJ0aWNpcGFudC4KCnN1Z2dlc3Rpb25zX3VzZV9mcmVxdWVuY3lfZGVza3RvcAoKOiAgIEFuc3dlciBmb3IgIkluIHRoZSBsYXN0IDI0IGhvdXJzLCBob3cgbWFueSB3b3JkIHN1Z2dlc3Rpb25zIGhhdmUgeW91IHVzZWQgd2hlbiB0eXBpbmcgb24gYSBkZXNrdG9wIGNvbXB1dGVyPyIuCgpzdWdnZXN0aW9uc191c2VfZnJlcXVlbmN5X21vYmlsZQoKOiAgIEFuc3dlciBmb3IgIkluIHRoZSBsYXN0IDI0IGhvdXJzLCBob3cgbWFueSB3b3JkIHN1Z2dlc3Rpb25zIGhhdmUgeW91IHVzZWQgd2hlbiB0eXBpbmcgb24gYSBtb2JpbGUgZGV2aWNlPyIuCgpjb250cm9sc19zYXRpc2ZhY3RvcnkKCjogICBBZ3JlZW1lbnQgZm9yICJUaGUgY29udHJvbHMgKGtleWJvYXJkIGFuZCB3b3JkIHN1Z2dlc3Rpb25zKSBhcmUgc2F0aXNmYWN0b3J5IGZvciB0aGUgY29tcGxldGlvbiBvZiB0aGUgdGFzay4iIFRoaXMgd2FzIHByb3ZpZGVkIGZvciBlYWNoIGNvbmRpdGlvbi4KCnN1Z2dlc3Rpb25fYWNjdXJhY3kKCjogICBBZ3JlZW1lbnQgZm9yICJUaGUgd29yZCBzdWdnZXN0aW9ucyBhcmUgYWNjdXJhdGUuIiBUaGlzIHdhcyBwcm92aWRlZCBmb3IgZWFjaCBjb25kaXRpb24uCgprZXlib2FyZF91c2VfZWZmaWNpZW5jeQoKOiAgIEFncmVlbWVudCBmb3IgIlRoZSB1c2Ugb2YgdGhlIGtleWJvYXJkIGlzIGVmZmljaWVudCBpbiB0aGlzIHRhc2suIiBUaGlzIHdhcyBwcm92aWRlZCBmb3IgZWFjaCBjb25kaXRpb24uCgpzdWdnZXN0aW9uX2Rpc3RyYWN0aW9uCgo6ICAgQWdyZWVtZW50IGZvciAiVGhlIHdvcmQgc3VnZ2VzdGlvbnMgYXJlIGRpc3RyYWN0aW5nLiIgVGhpcyB3YXMgcHJvdmlkZWQgZm9yIGVhY2ggY29uZGl0aW9uLgoKbWVudGFsX2RlbWFuZAoKOiAgIE5BU0EtVExYIG1lbnRhbCBkZW1hbmQgc2NhbGUuIFRoaXMgd2FzIHByb3ZpZGVkIGZvciBlYWNoIGNvbmRpdGlvbi4KCnBoeXNpY2FsX2RlbWFuZAoKOiAgIE5BU0EtVExYIHBoeXNpY2FsIGRlbWFuZCBzY2FsZS4gVGhpcyB3YXMgcHJvdmlkZWQgZm9yIGVhY2ggY29uZGl0aW9uLgoKdGVtcG9yYWxfZGVtYW5kCgo6ICAgTkFTQS1UTFggdGVtcG9yYWwgZGVtYW5kIHNjYWxlLiBUaGlzIHdhcyBwcm92aWRlZCBmb3IgZWFjaCBjb25kaXRpb24uCgpwZXJmb3JtYW5jZQoKOiAgIE5BU0EtVExYIHBlcmZvcm1hbmNlIHNjYWxlLiBUaGlzIHdhcyBwcm92aWRlZCBmb3IgZWFjaCBjb25kaXRpb24uCgplZmZvcnQKCjogICBOQVNBLVRMWCBwZXJmb3JtYW5jZSBzY2FsZS4gVGhpcyB3YXMgcHJvdmlkZWQgZm9yIGVhY2ggY29uZGl0aW9uLgoKZnJ1c3RyYXRpb24KCjogICBOQVNBLVRMWCBlZmZvcnQgc2NhbGUuIFRoaXMgd2FzIHByb3ZpZGVkIGZvciBlYWNoIGNvbmRpdGlvbi4KCnRvdGFsX3N1Z2dlc3Rpb25zCgo6ICAgVGhlIG51bWJlciBvZiBzdWdnZXN0aW9ucyBzaG93ZWQgdG8gdGhlIHBhcnRpY2lwYW50LiBUaGlzIGlzIGFsd2F5cyAxLgoKdG90YWxfYmx1cnJlZF9tZWFzdXJlZF90cmlhbHMKCjogICBSZWdpc3RlciB0aGUgbnVtYmVyIG9mIHRyaWFscyBkdXJpbmcgd2hpY2ggdGhlIHBhZ2Ugd2FzIHVuZm9jdXNlZCwgd2hpY2ggaXMgYW4gaW5kaWNhdGlvbiB0aGF0IHRoZSBwYXJ0aWNpcGFudCBtYXkgYmUgZGlzdHJhY3RlZC4KCnRvdGFsX2JsdXJyZWRfbWVhc3VyZWRfdHJpYWxzCgo6ICAgUmVnaXN0ZXIgdGhlIG51bWJlciBvZiB0eXBpbmcgdHJpYWxzIGR1cmluZyB3aGljaCB0aGUgcGFnZSB3YXMgdW5mb2N1c2VkLCB3aGljaCBpcyBhbiBpbmRpY2F0aW9uIHRoYXQgdGhlIHBhcnRpY2lwYW50IG1heSBiZSBkaXN0cmFjdGVkLgoKaXNfcnVuX2NvbXBsZXRlZAoKOiAgIFRSVUUgaWYgdGhlIGV4cGVyaW1lbnQgd2FzIGNvbXBsZXRlZC4KCmlzX3J1bl9tZWFzdXJlZAoKOiAgIE9uZSBwYXJ0aWNpcGFudCB3YXMgcmVtb3ZlZCBiZWNhdXNlIHRoZXkgbWlzdW5kZXJzdG9vZCB0aGV5IHRob3VnaCB0aGV5IGhhZCB0byB1c2UgYWxsIHN1Z2dlc3Rpb25zLiBBbm90aGVyIHBhcnRpY2lwYW50IHdhcyByZW1vdmVkIGJlY2F1c2UgdGhleSBwYXJ0aWNpcGF0ZWQgbW9yZSB0aGFuIG9uY2UuCgppc19wYXJ0aWNpcGFudF90cnVzdGVkCgo6ICAgVFJVRSBpZiB0aGUgcGFydGljaXBhbnQgcGFzc2VkIHRoZSBhdHRlbnRpb24gdGVzdC4gSWYgbm90LCB0aGUgcGFydGljaXBhbnQgaXMgZXhjbHVkZWQgZnJvbSBzdWJqZWN0aXZlIGFuZCBkZW1vZ3JhcGhpYyBhbmFseXNpcy4KCnJ1bl9yZWplY3Rpb25fcmVhc29uCgo6ICAgVGhlIHJlYXNvbiB3aHkgdGhlIHBhcnRpY2lwYW50IHdhcyBmdWxseSByZW1vdmVkIGZyb20gdGhlIGFuYWx5c2lzLiBUaGlzIGlzIG9ubHkgc2V0IGlmIGBpc19ydW5fbWVhc3VyZWRgIGlzIEZBTFNFLgoKYGBge3J9CnJ1bnMgPC0gcmVhZF9rZXl0aW1lX3J1bnMobWVhc3VyZWRfb25seSA9IEZBTFNFKQptZWFzdXJlZF9ydW5zIDwtIHJ1bnMgfD4gZmlsdGVyKGlzX3J1bl9tZWFzdXJlZCkKcnVucwpgYGAKCiMjIFRyaWFscwoKVGhpcyB0YWJsZSBjb250YWlucyBkYXRhIGxvZ3MgZnJvbSBhbGwgcGFydGljaXBhbnQgdHJpYWxzLgoKcGFydGljaXBhbnQKCjogICBQYXJ0aWNpcGFudCBJZGVudGlmaWVyLgoKYWNjdXJhY3kKCjogICBUaGUgd29yZCBzdWdnZXN0aW9uIGFjY3VyYWN5IGNvbmRpdGlvbi4KCmtleXRpbWUKCjogICBUaGUgZGV2aWNlIHVzZWQgZm9yIHRoZSBydW4uCgpzdWdnZXN0aW9uc190eXBlCgo6ICAgVGhlIHR5cGUgb2Ygd29yZCBzdWdnZXN0aW9ucyB1c2VkOiBpbmxpbmUgb3IgYmFyLgoKdHJpYWxfbnVtYmVyCgo6ICAgVGhlIG51bWJlciBvZiB0aGUgdHJpYWwuCgp0cmlhbF9pZAoKOiAgIFRoZSBpZGVudGlmaWVyIG9mIHRoZSB0cmlhbC4KCmlzX3ByYWN0aWNlCgo6ICAgSWYgdGhlIHRyaWFsIHdhcyBwYXJ0IG9mIGEgcHJhY3RpY2UgYmxvY2suCgpwaHJhc2UKCjogICBUaGUgcGhyYXNlIHBhcnRpY2lwYW50IGhhZCB0byB0eXBlIGR1cmluZyB0aGUgdHJpYWwuCgp0b3RhbF9jaGFycwoKOiAgIFRoZSBudW1iZXIgb2YgY2hhcmFjdGVyIGluIHRoZSBwaHJhc2UgdG8gdHlwZS4KCnRoZW9yZXRpY2FsX3NrcwoKOiAgIFRoZSBtYXhpbXVtIG51bWJlciBvZiBrZXlzdHJva2VzIHRoYXQgY291bGQgYmUgc2F2ZWQgdXNpbmcgd29yZCBzdWdnZXN0aW9ucyBkdXJpbmcgdGhlIHRyaWFsLgoKc3RhcnRfZGF0ZQoKOiAgIFRoZSBzdGFydCBkYXRlIGFuZCB0aW1lIG9mIHRoZSB0cmlhbCBpbiBbSVNPIDg2MDFdKGh0dHBzOi8vZW4ud2lraXBlZGlhLm9yZy93aWtpL0lTT184NjAxKSBmb3JtYXQuCgplbmRfZGF0ZQoKOiAgIFRoZSBlbmQgZGF0ZSBhbmQgdGltZSBvZiB0aGUgdHJpYWwgaW4gW0lTTyA4NjAxXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9JU09fODYwMSkgZm9ybWF0LgoKZHVyYXRpb24KCjogICBUaGUgZHVyYXRpb24gb2YgdGhlIHRyaWFsIGluIHNlY29uZHMuCgphY3R1YWxfc2tzCgo6ICAgVGhlIG51bWJlciBvZiBrZXlzdHJva2VzIHRoYXQgd2VyZSBzYXZlZCB1c2luZyB3b3JkIHN1Z2dlc3Rpb25zLiBUaGlzIG1heSBiZSBuZWdhdGl2ZSBpZiBpbmNvcnJlY3Qgc3VnZ2VzdGlvbnMgd2VyZSB1c2VkLgoKdG90YWxfa2V5X3N0cm9rZXMKCjogICBUaGUgbnVtYmVyIG9mIGtleSB0aGF0IHdlcmUgcHJlc3NlZCB0byBjb21wbGV0ZSB0aGUgdHJpYWwuCgp0b3RhbF9zdWdnZXN0aW9uX3VzZWQKCjogICBUaGUgbnVtYmVyIG9mIHN1Z2dlc3Rpb25zIHRoYXQgd2VyZSB1c2VkIGR1cmluZyB0aGUgdHJpYWwuCgp0b3RhbF9zdWdnZXN0aW9uX2Vycm9ycwoKOiAgIFRoZSBudW1iZXIgb2YgaW5jb3JyZWN0IHN1Z2dlc3Rpb25zIHRoYXQgd2VyZSB1c2VkIGR1cmluZyB0aGUgdHJpYWwuCgp0b3RhbF9yZW1vdmVkX21hbnVhbF9jaGFycwoKOiAgIFRoZSBudW1iZXIgb2YgY2hhcmFjdGVycyB0aGF0IHdlcmUgbWFudWFsbHkgZW50ZXJlZCB0aGVuIGRlbGV0ZWQgYnkgcGFydGljaXBhbnQgZHVyaW5nIHRoZSB0cmlhbC4KCnRvdGFsX3JlbW92ZWRfc3VnZ2VzdGlvbl9jaGFycwoKOiAgIFRoZSBudW1iZXIgb2YgY2hhcmFjdGVycyB0aGF0IHdlcmUgaW5zZXJ0ZWQgZnJvbSBhIHN1Z2dlc3Rpb24gdGhlbiBkZWxldGVkIGJ5IHBhcnRpY2lwYW50IGR1cmluZyB0aGUgdHJpYWwuCgp0b3RhbF9maW5hbF9tYW51YWxfY2hhcnMKCjogICBUaGUgbnVtYmVyIG9mIGNoYXJhY3RlcnMgZnJvbSB0aGUgZmluYWwgcGFydGljaXBhbnQgaW5wdXQgdGhhdCB3ZXJlIG1hbnVhbGx5IGluc2VydGVkLgoKdG90YWxfYWRkZWRfY2hhcnMKCjogICBUaGUgbnVtYmVyIG9mIGNoYXJhY3RlcnMgdGhhdCB3ZXJlIGluc2VydGVkIGR1cmluZyB0aGUgdHJpYWwgZWl0aGVyIGZyb20gc3VnZ2VzdGlvbiBvciBtYW51YWwgaW5wdXQgKHNvbWUgbWF5IGhhdmUgYmVlbiByZW1vdmVkIGxhdGVyKS4KCmRvZXNfdHJpYWxfaGF2ZV9lcnJvcnMKCjogICBUUlVFIGlmIGluY29ycmVjdCBjaGFyYWN0ZXJzIHdlcmUgaW5zZXJ0ZWQgYXQgc29tZSBwb2ludCBkdXJpbmcgdGhlIHRyaWFsICh0aGV5IGhhZCB0byBiZSByZW1vdmVkIGxhdGVyIHRvIHZhbGlkYXRlIHRoZSB0cmlhbCkuCgpwaHJhc2VfY29tcGxldGlvbl9zdGFydAoKOiAgIFRoZSBkYXRlIGFuZCB0aW1lIHdoZXJlIHRoZSBmaXJzdCBjaGFyYWN0ZXIgd2FzIGluc2VydGVkIGR1cmluZyB0aGUgdHJpYWwgKFtJU08gODYwMV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvSVNPXzg2MDEpIGZvcm1hdCkuCgpwaHJhc2VfY29tcGxldGlvbl9lbmQKCjogICBUaGUgZGF0ZSBhbmQgdGltZSB3aGVyZSB0aGUgcGhyYXNlIHdhcyBmdWxseSBjb21wbGV0ZWQgYW5kIGNvcnJlY3QgKmZvciB0aGUgZmlyc3QgdGltZSogKFtJU08gODYwMV0oaHR0cHM6Ly9lbi53aWtpcGVkaWEub3JnL3dpa2kvSVNPXzg2MDEpIGZvcm1hdCkuIE5vdGUgdGhhdCB0aGlzIGlzIGJlZm9yZSB0aGUgdHJpYWwgd2FzIHZhbGlkYXRlZC4KCnBocmFzZV9jb21wbGV0aW9uX2R1cmF0aW9uCgo6ICAgVGhlIHRpbWUgaXQgdG9vayBmb3IgdGhlIHBhcnRpY2lwYW50IHRvIGNvbXBsZXRlIHRoZSBwaHJhc2UgaW4gc2Vjb25kcy4KCnRvdGFsX2JsdXJfZXZlbnRzCgo6ICAgVGhlIG51bWJlciBvZiB0aW1lcyB0aGUgZXhwZXJpbWVudCB3ZWIgcGFnZSB3YXMgdW5mb2N1c2VkIGR1cmluZyB0aGUgdHJpYWwuCgp3YXNfdHJpYWxfYmx1cnJlZAoKOiAgIFRSVUUgaWYgdG90YWxfYmx1cl9ldmVudHMgXD4gMAoKdGhlb3JldGljYWxfa2V5X3NhdmluZwoKOiAgIFRoZSBtYXhpbXVtIGtleSBzdHJva2Ugc2F2aW5nIG9mIHRoZSB0cmlhbC4gVGhpcyBzaG91bGQgYmUgY2xvc2UgZnJvbSBgYWNjdXJhY3kuYAoKYWN0dWFsX2tleV9zYXZpbmdfbm9fZWRpdGluZwoKOiAgIFRoZSBrZXkgc3Ryb2tlIHNhdmluZyBleGNsdWRpbmcgZXJyb3JzLgoKYWN0dWFsX2tleV9zYXZpbmcKCjogICBUaGUga2V5IHN0cm9rZSBzYXZpbmcuCgpjcHMKCjogICBFbnRyeSBzcGVlZCBpbiBjaGFyYWN0ZXJzIHBlciBzZWNvbmQgKGkuZS4gYCh0b3RhbF9jaGFycyAtIDEpIC8gdHJpYWxfZHVyYXRpb25gKS4KCmlzX3J1bl9tZWFzdXJlZAoKOiAgIElmIHRoZSBydW4gaW5jbHVkaW5nIHRoaXMgdHJpYWwgd2FzIGluY2x1ZGVkIGluIG91ciBzdGF0aXN0aWNhbCBhbmFseXNpcy4KCmlzX3RyaWFsX21lYXN1cmVkCgo6ICAgSWYgdGhpcyB0cmlhbCB3YXMgaW5jbHVkZWQgaW4gb3VyIHN0YXRpc3RpY2FsIGFuYWx5c2lzLiBTZWUgYmVsb3cgZm9yIHJlYXNvbnMgdG8gZXhjbHVkZSBhIHRyaWFscyBmcm9tIG91ciBhbmFseXNpcy4KCnRyaWFsX3JlamVjdGlvbl9yZWFzb24KCjogICBUaGUgcmVhc29uIGZvciBleGNsdWRpbmcgdGhlIHRyaWFsIGZyb20gdGhlIGFuYWx5c2lzLiBUaGlzIGlzIG9ubHkgcHJvdmlkZWQgZm9yIHRyaWFscyB0aGF0IHdlcmUgZXhjbHVkZWQuCgpgYGB7cn0KdHJpYWxzIDwtIHJlYWRfa2V5dGltZV90cmlhbHMobWVhc3VyZWRfb25seSA9IEZBTFNFKQptZWFzdXJlZF90cmlhbHMgPC0gdHJpYWxzIHw+IGZpbHRlcihpc190cmlhbF9tZWFzdXJlZCkKdHJpYWxzCmBgYAoKIyMgRXZlbnRzCgpUaGlzIGxhcmdlIHRhYmxlIGNvbnRhaW5zIGFsbCBldmVudCBsb2dzIHJlY29yZGVkIGR1cmluZyBvdXIgZXhwZXJpbWVudC4gSXQgaW5jbHVkZXMgdGhlIGZvbGxvd2luZyBjb2x1bW5zOgoKcGFydGljaXBhbnQKCjogICBUaGUgcGFydGljaXBhbnQgaWRlbnRpZmllci4KCmFjY3VyYWN5Cgo6ICAgVGhlIGFjY3VyYWN5IGNvbmRpdGlvbi4KCmtleXRpbWUKCjogICBUaGUga2V5dGltZSBjb25kaXRpb24uCgpzdWdnZXN0aW9uc190eXBlCgo6ICAgVGhlIHR5cGUgb2Ygd29yZCBzdWdnZXN0aW9ucyB1c2VkOiBpbmxpbmUgb3IgYmFyLgoKdHJpYWxfaWQKCjogICBUaGUgdHJpYWwgaWRlbnRpZmllci4KCmV2ZW50X251bWJlcgoKOiAgIFRoZSBudW1iZXIgb2YgdGhlIGV2ZW50IGluIHRoZSB0cmlhbC4KCnR5cGUKCjogICBUaGUgdHlwZSBvZiB0aGUgZXZlbnQuIElOSVQ6IHRyaWFsIGluaXRpYWxpemF0aW9uLCBVUERBVEVfU1VHR0VTVElPTlM6IHVwZGF0ZSBvZiB0aGUgc3VnZ2VzdGlvbnMsIElOUFVUX0NIQVI6IG1hbnVhbCBpbnNlcnRpb24gb2YgYSBjaGFyYWN0ZXIgYnkgdGhlIHBhcnRpY2lwYW50IGF0IHRoZSBlbmQgb2YgdGhlaXIgaW5wdXQsIERFTEVURV9DSEFSOiByZW1vdmFsIG9mIHRoZSBsYXN0IGlucHV0IGNoYXJhY3RlciBieSB0aGUgcGFydGljaXBhbnQsIElOUFVUX1NVR0dFU1RJT046IGluc2VydGlvbiBvZiBhIHN1Z2dlc3Rpb24uCgppbnB1dAoKOiAgIFRoZSBjb250ZW50IG9mIHBhcnRpY2lwYW50J3MgaW5wdXQgYWZ0ZXIgdGhlIGV2ZW50LgoKaXNfaW5wdXRfY29ycmVjdAoKOiAgIElmIHRoZXJlIGlzIG5vIGVycm9ycyBpbiB0aGUgaW5wdXQgYWZ0ZXIgdGhlIGV2ZW50LgoKaXNfZXJyb3IKCjogICBJZiB0aGUgZXZlbnQgcmVzdWx0cyBmcm9tIGFuIGVycm9yLCBzdWNoIGFzIGNob29zaW5nIGFuIGluY29ycmVjdCBzdWdnZXN0aW9uIG9yIGVudGVyaW5nIGFuIGluY29ycmVjdCBjaGFyYWN0ZXIuCgphZGRlZF9pbnB1dAoKOiAgIFRoZSBjaGFyYWN0ZXJzIGFkZGVkIHRvIHRoZSBpbnB1dCBhcyBhIHJlc3VsdCBvZiB0aGUgZXZlbnQuCgpyZW1vdmVkX2lucHV0Cgo6ICAgVGhlIGNoYXJhY3RlcnMgcmVtb3ZlZCBmcm9tIHRoZSBpbnB1dCBhcyBhIHJlc3VsdCBvZiB0aGUgZXZlbnQuCgp0b3RhbF9hZGRlZF9jaGFycwoKOiAgIFRoZSBudW1iZXIgb2YgY2hhcmFjdGVycyBhZGRlZCBhcyByZXN1bHQgb2YgdGhlIGV2ZW50IChtYXkgYmUgbmVnYXRpdmUpLgoKcmVtYWluaW5nX2tleV9zdHJva2VzCgo6ICAgVGhlIG51bWJlciBvZiBjaGFyYWN0ZXIgbGVmdCB0byBjb21wbGV0ZSB0aGUgcGhyYXNlLgoKaXNfdGFyZ2V0X2NvbXBsZXRlZAoKOiAgIElmIHRoZSBwaHJhc2UgaXMgY29tcGxldGVkLgoKdGltZQoKOiAgIFRoZSBkYXRlIGFuZCB0aW1lIG9mIHRoZSBldmVudCAoW0lTTyA4NjAxXShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9JU09fODYwMSkgZm9ybWF0KS4KCmRpZmZfdGltZQoKOiAgIER1cmF0aW9uIGJldHdlZW4gdGhpcyBldmVudCBhbmQgdGhlIHByZXZpb3VzIG9uZS4KCnNjaGVkdWxlZF9hY3Rpb24KCjogICBUaGUgYWN0aW9uIHRoYXQgd2FzIHNjaGVkdWxlZCBmb3IgdGhpcyBldmVudCBkdWUgdG8gdGhlIGtleXRpbWUgY29uZGl0aW9uLgoKaXNfcnVuX21lYXN1cmVkCgo6ICAgSWYgdGhlIHJ1biBpbmNsdWRpbmcgdGhpcyBldmVudCB3YXMgaW5jbHVkZWQgaW4gb3VyIHN0YXRpc3RpY2FsIGFuYWx5c2lzLgoKaXNfdHJpYWxfbWVhc3VyZWQKCjogICBJZiB0aGUgdHJpYWwgaW5jbHVkaW5nIHRoaXMgZXZlbnQgd2FzIGluY2x1ZGVkIGluIG91ciBzdGF0aXN0aWNhbCBhbmFseXNpcy4KCmlzX2V2ZW50X21lYXN1cmVkCgo6ICAgSWYgdGhpcyBldmVudCB3YXMgaW5jbHVkZWQgaW4gb3VyIHN0YXRpc3RpY2FsIGFuYWx5c2lzLgoKZXZlbnRfcmVqZWN0aW9uX3JlYXNvbgoKOiAgIFRoZSByZWFzb24gZm9yIGV4Y2x1ZGluZyB0aGUgZXZlbnQgZnJvbSB0aGUgYW5hbHlzaXMuIFRoaXMgaXMgb25seSBwcm92aWRlZCBmb3IgZXZlbnRzIHRoYXQgd2VyZSBleGNsdWRlZC4KCnN1Z2dlc3Rpb24KCjogICBUaGUgc3VnZ2VzdGlvbiB0aGF0IHdhcyBpbnNlcnRlZCBhcyBhIHJlc3VsdCBvZiB0aGUgZXZlbnQuCgp0YXJnZXRfd29yZAoKOiAgIFRoZSBuZXh0IHdvcmQgdG8gdHlwZSBvciB0aGUgd29yZCBjdXJyZW50bHkgYmVpbmcgdHlwZWQuCgp0YXJnZXRfd29yZF9udW1iZXIKCjogICBUaGUgbnVtYmVyIG9mIHRoZSB0YXJnZXQgd29yZCBpbiB0aGUgcGhyYXNlLgoKIyBNb25pdG9yaW5nCgojIyBUb3RhbCBtZWFzdXJlZCBydW5zCgpgYGB7cn0KbWVhc3VyZWRfcnVucyB8PiBjb3VudChzdWdnZXN0aW9uc190eXBlKQpgYGAKCiMjIFRvdGFsIG1lYXN1cmVkIHRyaWFscwoKYGBge3J9Cm1lYXN1cmVkX3RyaWFscyB8PiBjb3VudChzdWdnZXN0aW9uc190eXBlKQpgYGAKCiMjIEJsdXJyZWQgVHJpYWxzCgpUaGUgZm9sbG93aW5nIHRyaWFscyB3ZXJlIHJlbW92ZWQgYmVjYXVzZSB0aGUgcGFydGljaXBhbnQgd2FzIGludGVycnVwdGVkICh0aGUgcGFnZSB3YXMgdW5mb2N1c2VkKS4KCmBgYHtyfQp0cmlhbHMgfD4gZmlsdGVyKGlzX3J1bl9tZWFzdXJlZCAmICFpc19wcmFjdGljZSkgfD4KICBjb3VudChzdWdnZXN0aW9uc190eXBlLCB3YXNfdHJpYWxfYmx1cnJlZCkgfD4KICBhZGRfY291bnQoc3VnZ2VzdGlvbnNfdHlwZSwgd3QgPSBuLCBuYW1lPSJ0b3RhbCIpCmBgYAoKIyMgUmVqZWN0ZWQgcGFydGljaXBhbnRzCgpJbiBhZGRpdGlvbiwgdGhlIGZvbGxvd2luZyBwYXJ0aWNpcGFudCB3ZXJlIGZ1bGx5IHJlbW92ZWQgZnJvbSBvdXIgYW5hbHlzaXMgYmVjYXVzZSB0aGV5IHVuZm9jdXNlZCB0aGUgcGFnZSBmb3IgbW9yZSB0aGFuIGByIE1BWF9CTFVSUkVEX1RSSUFMU2AgdHJpYWxzLgoKYGBge3J9CnJ1bnMgfD4KICBmaWx0ZXIoaXNfcnVuX2NvbXBsZXRlZCkgfD4KICBjb3VudChzdWdnZXN0aW9uc190eXBlLCBydW5fcmVqZWN0aW9uX3JlYXNvbikKYGBgCgojIyBVbnRydXN0ZWQgcGFydGljaXBhbnRzCgpGaW5hbGx5LCB0aGUgZm9sbG93aW5nIHBhcnRpY2lwYW50cyB3ZXJlIHJlbW92ZWQgZnJvbSB0aGUgZGVtb2dyYXBoaWNzIGFuZCBzdWJqZWN0aXZlIGFuYWx5c2lzIGJlY2F1c2UgdGhleSBkaWQgbm90IHBhc3MgdGhlIGF0dGVudGlvbiB0ZXN0IGR1cmluZyB0aGUgcXVlc3Rpb25uYWlyZSAoaG93ZXZlciB0aGVpciBvYmplY3RpdmUgZGF0YSB3YXMga2VwdCkuCgpgYGB7cn0KcnVucyB8PiBmaWx0ZXIoaXNfcnVuX21lYXN1cmVkKSB8PiBjb3VudChzdWdnZXN0aW9uc190eXBlLCBpc19wYXJ0aWNpcGFudF90cnVzdGVkKSB8PgogIGFkZF9jb3VudChzdWdnZXN0aW9uc190eXBlLCB3dCA9IG4sIG5hbWU9InRvdGFsIikKYGBgCgojIERhdGEgQW5hbHlzaXMKCi0gICBbT2JqZWN0aXZlIERhdGEgQW5hbHlzaXNdKC4va2V5dGltZS1vYmplY3RpdmUubmIuaHRtbCkKLSAgIFtTdWJqZWN0aXZlIERhdGEgQW5hbHlzaXNdKC4va2V5dGltZS1zdWJqZWN0aXZlLm5iLmh0bWwpCi0gICBbRGVtb2dyYXBoaWMgRGF0YSBBbmFseXNpc10oLi9rZXl0aW1lLWRlbW9ncmFwaGljcy5uYi5odG1sKQo=