dpnmw/agents
1
0
Fork 0
mirror of https://github.com/wshobson/agents.git synced 2026-03-18 09:37:15 +00:00
Code Issues Packages Projects Releases Wiki Activity

style: format all files with prettier

Browse Source
This commit is contained in:
Seth Hobson
2026-01-19 17:07:03 -05:00
parent 8d37048deb
commit 56848874a2
355 changed files with 15215 additions and 10241 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
plugins/data-engineering/skills/data-quality-frameworks/SKILL.md
View File

@@ -20,14 +20,14 @@ Production patterns for implementing data quality with Great Expectations, dbt t
### 1. Data Quality Dimensions
| Dimension | Description | Example Check |
|-----------|-------------|---------------|
| **Completeness** | No missing values | `expect_column_values_to_not_be_null` |
| **Uniqueness** | No duplicates | `expect_column_values_to_be_unique` |
| **Validity** | Values in expected range | `expect_column_values_to_be_in_set` |
| **Accuracy** | Data matches reality | Cross-reference validation |
| **Consistency** | No contradictions | `expect_column_pair_values_A_to_be_greater_than_B` |
| **Timeliness** | Data is recent | `expect_column_max_to_be_between` |
| Dimension | Description | Example Check |
| ---------------- | ------------------------ | -------------------------------------------------- |
| **Completeness** | No missing values | `expect_column_values_to_not_be_null` |
| **Uniqueness** | No duplicates | `expect_column_values_to_be_unique` |
| **Validity** | Values in expected range | `expect_column_values_to_be_in_set` |
| **Accuracy** | Data matches reality | Cross-reference validation |
| **Consistency** | No contradictions | `expect_column_pair_values_A_to_be_greater_than_B` |
| **Timeliness** | Data is recent | `expect_column_max_to_be_between` |
### 2. Testing Pyramid for Data
@@ -191,7 +191,7 @@ validations:
data_connector_name: default_inferred_data_connector_name
data_asset_name: orders
data_connector_query:
index: -1 # Latest batch
index: -1 # Latest batch
expectation_suite_name: orders_suite
action_list:
@@ -270,7 +270,8 @@ models:
- name: order_status
tests:
- accepted_values:
values: ['pending', 'processing', 'shipped', 'delivered', 'cancelled']
values:
["pending", "processing", "shipped", "delivered", "cancelled"]
- name: total_amount
tests:
@@ -566,6 +567,7 @@ if not all(r.passed for r in results.values()):
## Best Practices
### Do's
- **Test early** - Validate source data before transformations
- **Test incrementally** - Add tests as you find issues
- **Document expectations** - Clear descriptions for each test
@@ -573,6 +575,7 @@ if not all(r.passed for r in results.values()):
- **Version contracts** - Track schema changes
### Don'ts
- **Don't test everything** - Focus on critical columns
- **Don't ignore warnings** - They often precede failures
- **Don't skip freshness** - Stale data is bad data