Telemetry
What migkit keeps about how it behaved, what it never keeps, the default, and how to turn it off.
What is collected
how migkit behaved: the command, the engines, the ways chosen and why (as codes), timings and rates in buckets, workers allocated and used, progress and how far the estimate was off, error categories, retries and recoveries, how much was verified and to which guarantee, how a run ended and what the operator stepped in to do.
What is never collected
rows, values, samples, keys or digests of them, statements, schemas, the names of databases, tables, columns or accounts, addresses, paths, connection strings, credentials, logs, stack traces and error messages. Every field is a closed code, a bucket or a bounded number; anything else is refused where the event is made.
Why
to tell which ways work on which engines and shapes, how good the estimates are, and where runs fail or need a hand.
Where it goes
this machine first: events are kept under the state directory in sealed, append-only segments of a fixed size, each chained to the one before, the oldest dropped past the number or age the organisation's policy keeps (telemetry_keep, telemetry_days). Nothing leaves on its own: migkit telemetry push <bucket> sends the sealed segments, once each, to an S3-compatible bucket you run and name, signed in with that store's own credentials, never a key migkit carries; migkit telemetry receive <bucket> reads them back and checks every event again. With MIGKIT_EGRESS=none, or a policy saying telemetry_push: off, nothing is sent and the push says why.
Default
on: events are kept locally; a push happens only when you run it. migkit telemetry show prints exactly what is kept.
How to turn it off
migkit telemetry off, MIGKIT_TELEMETRY=off, MIGKIT_EGRESS=none, or an organisation policy file (MIGKIT_TELEMETRY_POLICY, /etc/migkit/policy.yaml by default) saying telemetry: off. Off stops keeping events and deletes those already kept, the sealed segments not yet pushed included.
Every field an event may have
An event is one of these kinds, each field a closed code, a bucket or a bounded number; a field not listed here, or a value outside its codes, is refused where the event is made.
| Event | Field | Kind | What it holds | Codes |
|---|---|---|---|---|
| run | version | one of | migkit's version | 0.2.0 |
| run | command | one of | the command | |
| run | engine | one of | the hop's engine | |
| run | target_engine | one of | the target's engine | |
| run | completion | one of | how it ended | done, same, different, could not verify, incomplete, usage, failed, refused, interrupted |
| run | error | one of | what stopped it, by kind | none, timeout, connection, permission, resource, not found, refused, usage, interrupted, data, stopped, internal |
| run | took | one of | how long it took | <1s, 1s+, 2s+, 4s+, 8s+, 16s+, 32s+, 64s+, 128s+, 256s+, 512s+, 1024s+, 2048s+, 4096s+, 8192s+, 16384s+, 32768s+, 65536s+, 131072s+, 262144s+, 524288s+, >=12d |
| run | seconds | number | how long it took | 0 to 10000000 |
| run | writes | flag | it was asked to write (--go, --apply) | |
| run | scheduled | flag | its schedule started it | |
| run | databases | one of | databases in the hop's scope | 0, 1-9, 1e1-1e2, 1e2-1e3, 1e3-1e4, 1e4-1e5, 1e5-1e6, 1e6-1e7, 1e7-1e8, 1e8-1e9, 1e9-1e10, 1e10-1e11, 1e11-1e12, >=1e12 |
| run | decisions | whole | ways chosen | 0 to 1000000 |
| run | fallbacks | whole | ways given up for another | 0 to 1000000 |
| run | options_set | whole | options the operator set | 0 to 1000 |
| run | dropped | whole | events dropped for a full queue | 0 to 1000000000 |
| decision | strategy | one of | the way chosen | |
| decision | reasons | some of | why, as codes | carried, left alone, row filter, column mapping, resume by chunk, resume by key, keyless, renamed, apart, exact, byte exact, consistent as of, online, masked on the way, access history, certificate checked, source position, into an older server, written in pieces, whole server, indexes built, pages proved, first choice, fell back, none holds, other |
| decision | guarantees | some of | the guarantees it reaches | correct:unproved, correct:counted, correct:proved, correct:byte exact, consistent:none, consistent:per range, consistent:as of one moment, idempotent:no, idempotent:converges, exactly once:at least once, exactly once:exactly once, resumable:starts over, resumable:goes on |
| decision | ranked | whole | ways that held | 0 to 1000 |
| decision | passed | whole | ways passed over | 0 to 1000 |
| decision | fallback | flag | it replaced a way chosen before | |
| decision | trial | flag | it ran on a bounded trial | |
| progress | phase | one of | the phase | discover, assess, configure, plan, schema, move, follow, verify, repair, cutover, rollback, report |
| progress | rows | one of | rows done | 0, 1-9, 1e1-1e2, 1e2-1e3, 1e3-1e4, 1e4-1e5, 1e5-1e6, 1e6-1e7, 1e7-1e8, 1e8-1e9, 1e9-1e10, 1e10-1e11, 1e11-1e12, >=1e12 |
| progress | bytes | one of | bytes done | 0, 1-9, 1e1-1e2, 1e2-1e3, 1e3-1e4, 1e4-1e5, 1e5-1e6, 1e6-1e7, 1e7-1e8, 1e8-1e9, 1e9-1e10, 1e10-1e11, 1e11-1e12, >=1e12 |
| progress | rate | one of | rows a second | 0, 1-9, 1e1-1e2, 1e2-1e3, 1e3-1e4, 1e4-1e5, 1e5-1e6, 1e6-1e7, 1e7-1e8, 1e8-1e9, 1e9-1e10, 1e10-1e11, 1e11-1e12, >=1e12 |
| progress | took | one of | how long it has run | <1s, 1s+, 2s+, 4s+, 8s+, 16s+, 32s+, 64s+, 128s+, 256s+, 512s+, 1024s+, 2048s+, 4096s+, 8192s+, 16384s+, 32768s+, 65536s+, 131072s+, 262144s+, 524288s+, >=12d |
| progress | eta_off | number | actual time over the estimate | 0 to 1000 |
| progress | done | one of | how much of it is done | 0, <50%, <90%, <99%, <100%, 100% |
| resources | phase | one of | the phase | discover, assess, configure, plan, schema, move, follow, verify, repair, cutover, rollback, report |
| resources | workers_allocated | whole | workers allocated | 0 to 100000 |
| resources | workers_used | whole | workers that ran | 0 to 100000 |
| resources | connections | whole | connections held | 0 to 100000 |
| resources | memory | one of | bytes of memory the workers hold | 0, 1-9, 1e1-1e2, 1e2-1e3, 1e3-1e4, 1e4-1e5, 1e5-1e6, 1e6-1e7, 1e7-1e8, 1e8-1e9, 1e9-1e10, 1e10-1e11, 1e11-1e12, >=1e12 |
| resources | cpus | whole | processors on this machine | 0 to 10000 |
| recovery | phase | one of | the phase | discover, assess, configure, plan, schema, move, follow, verify, repair, cutover, rollback, report |
| recovery | error | one of | the failure, by kind | none, timeout, connection, permission, resource, not found, refused, usage, interrupted, data, stopped, internal |
| recovery | fault | one of | the class of failure | connectivity, auth, permission, expired_credential, quota, cpu_pressure, memory_pressure, disk_pressure, lock, packet, statement_timeout, orphaned, duplicate, uncertain_commit, interruption, failover, stale_checkpoint, inconsistent_view, lost_history, schema_change, lossy_conversion, malformed_data, corrupted_evidence, dependency, external_impact, unknown |
| recovery | path | one of | the action the decision engine chose | reconcile, proceed, pause, throttle, retry, replan, isolate, probe, stop |
| recovery | action | one of | what was done | retry, resume, fallback, autotune, reconcile, gave up |
| recovery | outcome | one of | how it ended | recovered, failed, blocked |
| recovery | attempts | whole | attempts | 0 to 10000 |
| coverage | phase | one of | the phase | discover, assess, configure, plan, schema, move, follow, verify, repair, cutover, rollback, report |
| coverage | verified | one of | the scope verified | 0, <50%, <90%, <99%, <100%, 100% |
| coverage | guarantees | some of | the guarantees reached | correct:unproved, correct:counted, correct:proved, correct:byte exact, consistent:none, consistent:per range, consistent:as of one moment, idempotent:no, idempotent:converges, exactly once:at least once, exactly once:exactly once, resumable:starts over, resumable:goes on |
| coverage | completion | one of | the verdict | done, same, different, could not verify, incomplete, usage, failed, refused, interrupted |
| coverage | units | one of | units compared | 0, 1-9, 1e1-1e2, 1e2-1e3, 1e3-1e4, 1e4-1e5, 1e5-1e6, 1e6-1e7, 1e7-1e8, 1e8-1e9, 1e9-1e10, 1e10-1e11, 1e11-1e12, >=1e12 |
| unit | phase | one of | the phase | discover, assess, configure, plan, schema, move, follow, verify, repair, cutover, rollback, report |
| unit | scope | one of | what kind of unit it is | table, database, range |
| unit | unit | opaque | the unit, as a salted hash | |
| unit | outcome | one of | how the unit compared | ok, skip, warn, diff, error |
| unit | guarantees | some of | the guarantees it reached | correct:unproved, correct:counted, correct:proved, correct:byte exact, consistent:none, consistent:per range, consistent:as of one moment, idempotent:no, idempotent:converges, exactly once:at least once, exactly once:exactly once, resumable:starts over, resumable:goes on |
| intervention | phase | one of | the phase | discover, assess, configure, plan, schema, move, follow, verify, repair, cutover, rollback, report |
| intervention | did | one of | what they did | paused, resumed, cancelled, approved, overrode, retried by hand, repaired by hand, asked, answered, bundled |
| intervention | question | one of | the question migkit put to them | source, target, password, goal, write, target_rows, grant, switch |
| crash | command | one of | the command | |
| crash | phase | one of | the phase it worked in | discover, assess, configure, plan, schema, move, follow, verify, repair, cutover, rollback, report |
| crash | error | one of | what stopped it, by kind | none, timeout, connection, permission, resource, not found, refused, usage, interrupted, data, stopped, internal |