Validate JSON data and check it against a schema

Data Validator — Schema Validation, Format Checking, and Data Quality Analysis in One Pass

Data validation failures in production systems share a common characteristic: the invalid data was accepted at some boundary — an API endpoint, a file upload form, a database insert — without being checked, and then propagated through multiple downstream systems before surfacing as a runtime error in a system that was never designed to handle the malformed input. A phone number stored without its country code causes international SMS delivery failures months after the record was created. An email address stored with a trailing space causes login failures when the authentication system trims input but the storage system did not. A date stored in M/D/YYYY format causes parsing errors in systems that expect ISO 8601 YYYY-MM-DD. Each of these failures has the same fix: validation at the point of entry that rejects invalid data before it enters the system. The data validator applies configurable validation rules to data files before they enter any downstream pipeline, identifying every violation in a single pass rather than discovering them one at a time through runtime errors.

JSON Schema validation provides declarative, machine-readable specifications for valid data structures. A JSON Schema defines the expected shape of a JSON document: which properties are required, which are optional, the data type of each property (string, number, integer, boolean, array, object), the format constraints on string properties (email, date-time, URI, UUID), the numeric range constraints on number properties (minimum, maximum), the length constraints on string and array properties (minLength, maxLength, minItems, maxItems), and the enumerated set of valid values for properties with limited value sets. Validating a JSON payload against its schema produces an explicit error report identifying every violation — missing required field, wrong type, value out of range, unrecognized enum value — with the exact property path of each violation. This error report is the difference between a system that silently accepts invalid data and one that rejects it with actionable feedback.

Statistical data quality analysis reveals patterns of invalidity that schema validation cannot detect because the invalid values are technically well-formed. A phone number column where 23% of values are "555-555-5555" is technically valid strings but likely contains test data that was never replaced with real phone numbers. An age column where values include 0 and 999 are valid integers but outside the plausible range for any real customer. A revenue column where 15% of values are exactly 0 suggests either legitimate zero-revenue customers or a data collection failure that substituted null with zero — a difference with significant analytical implications. Data quality analysis reports completeness (percentage of non-null values), uniqueness (percentage of distinct values, flagging unexpected duplicates), distribution summary (min, max, mean, median, standard deviation for numeric columns), and top-value frequency for categorical columns — surfacing these patterns for human judgment before data enters any analytical workflow.