tthhanh commited on
Commit
097ec6e
·
1 Parent(s): b3d9fda

docs: remove archived planning documents

Browse files
docs/01-docs-index.md CHANGED
@@ -18,15 +18,6 @@ Read in this order if you want to understand and run the current project:
18
 
19
  - [11-overview-demo-video-transcript.md](11-overview-demo-video-transcript.md)
20
 
21
- ## Planning / Historical Design Notes
22
-
23
- These files are useful context, but they are not the source of truth for current runtime commands:
24
-
25
- - [90-archive-implementation-plan.md](90-archive-implementation-plan.md)
26
- - [91-archive-coach-analysis-implementation-plan.md](91-archive-coach-analysis-implementation-plan.md)
27
- - [92-archive-coach-model-data-plan.md](92-archive-coach-model-data-plan.md)
28
- - [99-archive-pozify-project-documentation.md](99-archive-pozify-project-documentation.md)
29
-
30
  ## Notes
31
 
32
  - Current coach-summary cloud runtime default: `Qwen/Qwen3-14B`
 
18
 
19
  - [11-overview-demo-video-transcript.md](11-overview-demo-video-transcript.md)
20
 
 
 
 
 
 
 
 
 
 
21
  ## Notes
22
 
23
  - Current coach-summary cloud runtime default: `Qwen/Qwen3-14B`
docs/90-archive-implementation-plan.md DELETED
@@ -1,128 +0,0 @@
1
- # Pozify Implementation Plan
2
-
3
- Status: historical implementation plan.
4
-
5
- Use [../README.md](../README.md) and [01-docs-index.md](01-docs-index.md) for the current runtime
6
- workflow and operational commands.
7
-
8
- This plan breaks Pozify into implementation workstreams that can be developed independently while preserving the current JSON data contracts.
9
-
10
- ## Phase 1: Foundation And Contracts
11
-
12
- Goal: make the mocked pipeline production-shaped before replacing mock logic.
13
-
14
- Deliverables:
15
-
16
- - Stable JSON contracts in `src/pozify/contracts.py`.
17
- - Artifact persistence under `runs/<run_id>/`.
18
- - Contract validation and unit tests.
19
- - Clear step boundaries for replacing mock implementations.
20
- - Seed knowledge card structure.
21
-
22
- ## Phase 2: Video Intake And Quality Gate
23
-
24
- Goal: accept user videos, extract reliable metadata, and reject videos that cannot be analyzed safely.
25
-
26
- Deliverables:
27
-
28
- - OpenCV-backed video metadata extraction.
29
- - Duration, FPS, brightness, blur, and basic frame-count checks.
30
- - Pose-quality-aware analysis gate.
31
- - User-facing quality warnings.
32
- - Capture guidance for failed videos.
33
-
34
- ## Phase 3: Pose Extraction And Cleaning
35
-
36
- Goal: convert videos into a clean 17-point 3D pose sequence.
37
-
38
- Deliverables:
39
-
40
- - MediaPipe Pose Landmarker integration.
41
- - Per-frame landmark persistence.
42
- - Pose quality scoring.
43
- - Smoothing and interpolation.
44
- - Coordinate normalization.
45
-
46
- ## Phase 4: Exercise Router
47
-
48
- Goal: classify the video as squat, push-up, shoulder press, or unknown.
49
-
50
- Deliverables:
51
-
52
- - Pose feature extraction for 30-frame windows.
53
- - Baseline classifier.
54
- - Fine-tuned small temporal classifier.
55
- - Confidence-weighted aggregation.
56
- - Manual fallback when confidence is low.
57
-
58
- ## Phase 5: Rep Segmentation
59
-
60
- Goal: split each exercise into clean reps using transparent state machines.
61
-
62
- Deliverables:
63
-
64
- - Squat rep counter.
65
- - Push-up rep counter.
66
- - Shoulder press rep counter.
67
- - Partial-rep handling.
68
- - Rep boundary visualization/debug output.
69
-
70
- ## Phase 6: Rep Analysis And Variation Detection
71
-
72
- Goal: compute meaningful metrics per rep and separate valid variations from issues.
73
-
74
- Deliverables:
75
-
76
- - Common rep metrics.
77
- - Exercise-specific metrics.
78
- - Variation detection rules.
79
- - Aggregate trends such as fatigue and decreasing ROM.
80
-
81
- ## Phase 7: Frame-Level Issue Markers
82
-
83
- Goal: identify exact issue intervals and attach evidence to each issue.
84
-
85
- Deliverables:
86
-
87
- - Rule-based frame-level issue scores.
88
- - Consecutive-frame thresholding.
89
- - Issue interval grouping.
90
- - Optional temporal issue classifier.
91
- - Evidence payloads for each marker.
92
-
93
- ## Phase 8: Annotated Video And UI
94
-
95
- Goal: make the evidence visible and easy to inspect.
96
-
97
- Deliverables:
98
-
99
- - Skeleton overlay renderer.
100
- - Red/amber highlights during issue intervals.
101
- - Rep counter overlay.
102
- - Issue thumbnails.
103
- - Gradio report layout with summary, metrics, reps, issues, and JSON evidence tabs.
104
-
105
- ## Phase 9: Grounded Coach Summary
106
-
107
- Goal: generate concise coaching feedback from structured evidence, not model memory.
108
-
109
- Deliverables:
110
-
111
- - Exercise, variation, issue, goal, and safety knowledge cards.
112
- - Deterministic retrieval.
113
- - SLM prompt contract.
114
- - Summary verifier.
115
- - Conservative fallback summary.
116
-
117
- ## Phase 10: Dataset, Training, Evaluation, And Demo
118
-
119
- Goal: create a defensible small-model story and a reliable hackathon demo.
120
-
121
- Deliverables:
122
-
123
- - Dataset ingestion scripts.
124
- - Custom data collection guide.
125
- - Training pipeline for the exercise router.
126
- - Evaluation metrics.
127
- - Demo videos.
128
- - README and deployment instructions.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/91-archive-coach-analysis-implementation-plan.md DELETED
@@ -1,183 +0,0 @@
1
- # Pozify Coach Analysis Expansion Plan
2
-
3
- Status: historical implementation plan for the coach-analysis branch work.
4
-
5
- Use [../README.md](../README.md) and [01-docs-index.md](01-docs-index.md) for the current runtime
6
- workflow and operational commands.
7
-
8
- ## Objective
9
-
10
- Extend Pozify's grounded coach-summary pipeline with a dataset-ready knowledge layer so the app can incorporate richer exercise expertise without turning the product into a generic chatbot.
11
-
12
- The target outcome is:
13
-
14
- - Keep summary generation grounded in structured artifacts.
15
- - Expand exercise knowledge through deterministic retrieval.
16
- - Preserve verifier safety rules and current UI/report interfaces.
17
- - Create a clean path for later ingesting open datasets such as `hasaneyldrm/exercises-dataset`.
18
-
19
- ## Why This Fits The Current Codebase
20
-
21
- Pozify already has the right runtime shape:
22
-
23
- - `src/pozify/pipeline.py` orchestrates the end-to-end flow.
24
- - `src/pozify/knowledge_cards.py` provides deterministic card retrieval.
25
- - `src/pozify/slm/prompting.py` injects knowledge cards into the SLM prompt.
26
- - `src/pozify/steps/verifier.py` rejects unsafe or ungrounded output.
27
-
28
- That means the highest-leverage change is not a chat finetune. It is a stronger knowledge-source layer that can feed better grounded cards into the existing summary step.
29
-
30
- ## Recommended Data Strategy
31
-
32
- ### Primary data source
33
-
34
- Use `hasaneyldrm/exercises-dataset` as the main source for exercise-grounding expansion because it maps naturally to Pozify's current `knowledge_cards` architecture:
35
-
36
- - exercise names
37
- - target muscles
38
- - equipment
39
- - execution cues
40
- - standardized instructions
41
-
42
- ### Secondary sources
43
-
44
- Use conversational datasets such as:
45
-
46
- - `onurSakar/GYM-Exercise`
47
- - `HazSylvia/Fitness_Unformatted`
48
- - `chibbss/fitness-chat-prompt-completion-dataset`
49
-
50
- only later, and only for:
51
-
52
- - coach voice calibration
53
- - response phrasing
54
- - optional supervised examples for tone
55
-
56
- These should not be the primary source for grounded issue reasoning.
57
-
58
- ## Implementation Steps
59
-
60
- ### Step 1. Plan and branch setup
61
-
62
- Deliverables:
63
-
64
- - This plan document.
65
- - New branch `ht/coach-analysis`.
66
-
67
- Commit:
68
-
69
- - `docs: add coach-analysis implementation plan`
70
-
71
- ### Step 2. Add dataset-ready card pack loading
72
-
73
- Goal:
74
-
75
- Allow Pozify to load extra knowledge cards from local JSON packs while keeping built-in cards as safe defaults.
76
-
77
- Deliverables:
78
-
79
- - A local card-pack loader in `src/pozify/knowledge_cards.py` or adjacent module.
80
- - Deterministic merge rules:
81
- - built-in cards always exist
82
- - external packs may add new cards
83
- - external packs may enrich known labels without breaking retrieval order
84
- - Environment-based configuration for optional pack paths.
85
-
86
- Why:
87
-
88
- This lets us transform open datasets into card packs offline and plug them into the existing summary pipeline without changing the model contract.
89
-
90
- Commit:
91
-
92
- - `feat: load external coach knowledge card packs`
93
-
94
- ### Step 3. Add a sample coach-analysis pack
95
-
96
- Goal:
97
-
98
- Ship one real example pack that demonstrates how open dataset knowledge will be represented inside Pozify.
99
-
100
- Deliverables:
101
-
102
- - A checked-in JSON pack under `data/knowledge_cards/`.
103
- - Enriched entries for existing exercises such as:
104
- - squat
105
- - push-up
106
- - shoulder press
107
- - Additional factual fields expressed through current card schema:
108
- - execution cues
109
- - equipment context
110
- - target-muscle context
111
-
112
- Commit:
113
-
114
- - `feat: add sample external coach knowledge pack`
115
-
116
- ### Step 4. Surface card provenance in artifacts
117
-
118
- Goal:
119
-
120
- Make it observable when coach summaries used built-in cards versus dataset-backed cards.
121
-
122
- Deliverables:
123
-
124
- - Final report artifact metadata indicating:
125
- - whether external packs were loaded
126
- - which pack files were used
127
- - how many retrieved cards came from external packs
128
-
129
- Commit:
130
-
131
- - `feat: expose coach knowledge provenance in artifacts`
132
-
133
- ### Step 5. Add tests for deterministic retrieval and pack loading
134
-
135
- Goal:
136
-
137
- Protect the summary pipeline from nondeterministic knowledge changes.
138
-
139
- Deliverables:
140
-
141
- - Unit tests for:
142
- - valid external pack parsing
143
- - deterministic merge behavior
144
- - duplicate-card override rules
145
- - retrieval by label with external packs enabled
146
- - pipeline artifact provenance
147
-
148
- Commit:
149
-
150
- - `test: cover external coach knowledge retrieval`
151
-
152
- ### Step 6. Validate end-to-end behavior
153
-
154
- Goal:
155
-
156
- Confirm the current app still runs and contract tests still pass.
157
-
158
- Deliverables:
159
-
160
- - Passing targeted tests:
161
- - `tests.test_coach_summary`
162
- - `tests.test_pipeline_contracts`
163
- - No regressions in current UI/report flow.
164
-
165
- Commit:
166
-
167
- - `test: validate coach-analysis pipeline integration`
168
-
169
- ## Follow-On Work After This Branch
170
-
171
- These items are intentionally out of scope for the first implementation pass but become straightforward after the card-pack layer exists:
172
-
173
- 1. Build a dataset transformer that converts `hasaneyldrm/exercises-dataset` rows into Pozify card packs.
174
- 2. Add a small curated evaluation set of artifact-to-summary pairs.
175
- 3. Fine-tune a small instruct model on Pozify-native JSON-to-summary examples.
176
- 4. Optionally add a separate chat/Q&A mode using the conversational datasets.
177
-
178
- ## Success Criteria
179
-
180
- - Pozify still generates grounded summaries from structured artifacts.
181
- - The app can ingest richer exercise knowledge through deterministic local packs.
182
- - Retrieval remains explainable and testable.
183
- - Verifier and fallback behavior remain unchanged in safety-sensitive cases.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/92-archive-coach-model-data-plan.md DELETED
@@ -1,660 +0,0 @@
1
- # Pozify Coach Model Data Plan
2
-
3
- Status: planning and architecture note.
4
-
5
- This file is useful for model and dataset strategy, but it is not the source of truth for the
6
- current run/deploy commands. Use [../README.md](../README.md) and
7
- [30-coach-modal-training.md](30-coach-modal-training.md) for the current workflow.
8
-
9
- ## Objective
10
-
11
- Define a practical data and modeling plan for improving Pozify's coach summary and coach-style guidance using open datasets and the current codebase.
12
-
13
- This document answers:
14
-
15
- - Which public datasets are actually useful for Pozify's current problem.
16
- - Whether to use RAG, SFT, or both.
17
- - How those choices fit the current pipeline, artifacts, verifier, and UI.
18
- - A concrete implementation roadmap that does not break the current app architecture.
19
-
20
- ## Current Codebase Fit
21
-
22
- Pozify today is not a generic fitness chatbot. It is a structured video-analysis pipeline that ends with a grounded coach summary:
23
-
24
- - `video_qc` -> `pose_landmarker` -> `exercise_classifier` -> `rep_analysis` -> `variation` -> `issue_markers` -> `coach_summary` -> `verifier`
25
- - The coach summary step already consumes structured evidence and retrieved knowledge cards.
26
- - The verifier already rejects summaries that invent issues, overcorrect valid variations, or make unsafe claims.
27
-
28
- Relevant modules:
29
-
30
- - `src/pozify/pipeline.py`
31
- - `src/pozify/steps/coach_summary.py`
32
- - `src/pozify/slm/prompting.py`
33
- - `src/pozify/knowledge_cards.py`
34
- - `src/pozify/steps/verifier.py`
35
-
36
- That means any dataset strategy must be evaluated against the real task:
37
-
38
- > Generate safe, grounded, structured coaching language from Pose/Rep/Variation/Issue JSON artifacts.
39
-
40
- This is **not** the same task as:
41
-
42
- - answering open-ended fitness questions,
43
- - free-form nutrition coaching,
44
- - broad lifestyle coaching,
45
- - or generic health chat.
46
-
47
- ## Recommendation Summary
48
-
49
- ### Best near-term strategy
50
-
51
- Use a **hybrid approach**:
52
-
53
- 1. **Expand knowledge retrieval first** using a curated exercise knowledge base.
54
- 2. **Keep coach summary grounded on structured Pozify artifacts.**
55
- 3. **Use lightweight SFT only after you have a Pozify-native training set** built from your own analysis artifacts and target summaries.
56
-
57
- ### What not to do first
58
-
59
- Do **not** directly fine-tune the current coach-summary model on broad fitness chat datasets and expect it to improve the grounded summary step.
60
-
61
- That would likely:
62
-
63
- - make the model more conversational,
64
- - but also make it more likely to hallucinate,
65
- - talk beyond the provided JSON,
66
- - and fight against the verifier.
67
-
68
- ### Strong recommendation
69
-
70
- For the current codebase, the best order is:
71
-
72
- 1. **RAG / retrieval upgrade**
73
- 2. **Pozify-native supervised dataset creation**
74
- 3. **LoRA / SFT on the Pozify-native task**
75
- 4. **Optional separate chatbot mode** trained on public chat/Q&A datasets
76
-
77
- ## Dataset-by-Dataset Assessment
78
-
79
- ### 1. `onurSakar/GYM-Exercise`
80
-
81
- Source notes:
82
-
83
- - Hugging Face dataset page shows `1.66k` rows and a single text field formatted in Llama-style instruction format.
84
- - The visible examples include fitness Q&A, but also broader wellness topics such as environmental stressors, work-life balance, mental clarity, and social well-being.
85
-
86
- Fit for Pozify:
87
-
88
- - Good for **general fitness coach tone**.
89
- - Weak fit for **grounded rep-by-rep coach summary generation**.
90
- - Weak fit for **variation-vs-issue reasoning** tied to structured pose artifacts.
91
-
92
- Risks:
93
-
94
- - Topic drift beyond movement coaching.
95
- - May encourage generic answer patterns instead of artifact-conditioned reasoning.
96
- - Could increase verifier failures because the model may mention things outside `issue_markers.json`.
97
-
98
- Best use in Pozify:
99
-
100
- - Use only as **auxiliary style data**.
101
- - Filter to movement-technique and workout-guidance samples.
102
- - Do not use raw as the main SFT dataset for `coach_summary`.
103
-
104
- Verdict:
105
-
106
- - **Useful later**
107
- - **Not the primary dataset for the current summary pipeline**
108
-
109
- ### 2. `HazSylvia/Fitness_Unformatted`
110
-
111
- Source notes:
112
-
113
- - Hugging Face page shows `928` rows.
114
- - Fields appear as `Human` and `Assistant`.
115
- - Examples are cleaner fitness Q&A than the dataset above and closer to gym onboarding and workout guidance.
116
-
117
- Fit for Pozify:
118
-
119
- - Better than `onurSakar/GYM-Exercise` for conversational coach tone.
120
- - Still not tightly aligned with Pozify's structured JSON-to-summary task.
121
-
122
- Risks:
123
-
124
- - Still mostly free-form Q&A, not evidence-grounded explanation.
125
- - Lacks explicit inputs like `rep_analysis`, `variation`, `issue_markers`.
126
-
127
- Best use in Pozify:
128
-
129
- - Use as **secondary SFT data for tone/voice**.
130
- - Convert to JSONL and filter for exercise technique, beginner coaching, and workout-planning responses.
131
- - Consider this dataset for a future `Ask Pozify` chat mode.
132
-
133
- Verdict:
134
-
135
- - **Good for coach voice**
136
- - **Not enough by itself for the current pipeline**
137
-
138
- ### 3. `chibbss/fitness-chat-prompt-completion-dataset`
139
-
140
- Source notes:
141
-
142
- - Hugging Face page shows `245` rows.
143
- - Has `instruction` and `output` fields.
144
- - Content appears to focus on healthy habits, routines, stress, sleep, and lifestyle guidance.
145
-
146
- Fit for Pozify:
147
-
148
- - Good format for instruction tuning.
149
- - Too small and too broad to be the main fine-tuning source for the current task.
150
-
151
- Risks:
152
-
153
- - Small size.
154
- - Lifestyle-heavy answers may not improve form-review summaries.
155
- - Could push the model toward broad wellness advice instead of evidence-based movement coaching.
156
-
157
- Best use in Pozify:
158
-
159
- - Optional augmentation for tone.
160
- - Better suited to a future “habit coach” or “ask a coach” feature than to movement-summary generation.
161
-
162
- Verdict:
163
-
164
- - **Supplementary only**
165
-
166
- ### 4. `hasaneyldrm/exercises-dataset`
167
-
168
- Based on your description, this is the most strategically valuable dataset for Pozify right now.
169
-
170
- Fit for Pozify:
171
-
172
- - High fit for retrieval and knowledge expansion.
173
- - It maps well to Pozify's current `knowledge_cards` architecture.
174
- - It improves factual consistency for exercise descriptions, target muscles, equipment, and execution cues.
175
-
176
- Why it fits the codebase:
177
-
178
- - Pozify already retrieves deterministic cards in `src/pozify/knowledge_cards.py`.
179
- - The prompt builder already injects knowledge cards into the model input.
180
- - The current gap is not “more chat data”; the gap is “richer, broader, more standardized exercise knowledge.”
181
-
182
- Best use in Pozify:
183
-
184
- - Use this as the basis for a **local knowledge base** first.
185
- - Convert exercise rows into:
186
- - exercise cards,
187
- - variation cards,
188
- - equipment-aware cues,
189
- - and optional goal-specific cue overlays.
190
- - Later, optionally index it in FAISS/Chroma for semantic retrieval in a separate chat flow.
191
-
192
- Verdict:
193
-
194
- - **Best immediate dataset to integrate**
195
-
196
- ### 5. `strova-ai/fitness-tracker-dataset`
197
-
198
- Source notes:
199
-
200
- - Hugging Face page describes it as a synthetic wearable/activity dataset.
201
- - It contains tabular metrics such as age, gender, height, weight, steps, heart rate, calories, distance, and activity.
202
- - The page also shows dataset-viewer schema issues, which suggests ingestion may need cleanup.
203
-
204
- Fit for Pozify:
205
-
206
- - Useful for personalization experiments.
207
- - Weak fit for current video-form summary generation.
208
-
209
- Why it does not fit immediately:
210
-
211
- - Pozify's current app does not reason from wearable timeseries.
212
- - The current `UserProfile` contract is small and categorical.
213
- - There is no current component that transforms tabular physiology into grounded motion advice.
214
-
215
- Best use in Pozify:
216
-
217
- - Optional later-stage personalization module.
218
- - Useful for future “today's activity context” or “session recommendation” features.
219
- - Not a priority for the current coach summary pipeline.
220
-
221
- Verdict:
222
-
223
- - **Future personalization dataset, not Phase 1**
224
-
225
- ## What This Means For Pozify
226
-
227
- ### The current task is not generic chat SFT
228
-
229
- Pozify's core output is:
230
-
231
- - bounded by `exercise_classification.json`
232
- - bounded by `rep_analysis.json`
233
- - bounded by `variation.json`
234
- - bounded by `issue_markers.json`
235
- - conditioned by knowledge cards
236
- - checked by a verifier
237
-
238
- So the most valuable training examples are not public generic fitness Q&A.
239
-
240
- The most valuable examples look like this:
241
-
242
- ```json
243
- {
244
- "analysis_json": {
245
- "user_profile": {},
246
- "exercise_classification": {},
247
- "rep_analysis": {},
248
- "variation": {},
249
- "issue_markers": {}
250
- },
251
- "retrieved_knowledge_cards": [],
252
- "ideal_coach_summary": {
253
- "summary": "",
254
- "what_you_did": [],
255
- "what_looked_good": [],
256
- "what_changed_across_reps": [],
257
- "valid_variation_vs_issue": [],
258
- "top_fixes": [],
259
- "next_session_plan": [],
260
- "confidence_notes": []
261
- }
262
- }
263
- ```
264
-
265
- This is already much closer to the current runtime prompt contract than any public chat dataset you listed.
266
-
267
- ## Recommended Strategy
268
-
269
- ## Option A: Retrieval-first upgrade
270
-
271
- ### Why this is the best first move
272
-
273
- It fits the current architecture with the least risk.
274
-
275
- It improves:
276
-
277
- - factual exercise guidance,
278
- - coaching cue coverage,
279
- - equipment-specific instructions,
280
- - and future exercise expansion,
281
-
282
- without retraining the model first.
283
-
284
- ### How to use `hasaneyldrm/exercises-dataset`
285
-
286
- Convert exercise rows into a normalized internal artifact such as:
287
-
288
- ```json
289
- {
290
- "card_id": "exercise:barbell_bench_press",
291
- "card_type": "exercise",
292
- "labels": ["barbell_bench_press", "bench_press"],
293
- "title": "Barbell Bench Press",
294
- "summary": "Compound upper-body press.",
295
- "target_muscles": ["chest"],
296
- "secondary_muscles": ["triceps", "front_delts"],
297
- "equipment": ["barbell", "bench"],
298
- "instructions": [
299
- "Set the shoulder blades before unracking.",
300
- "Lower with control to a repeatable touch point.",
301
- "Press while keeping wrists stacked."
302
- ]
303
- }
304
- ```
305
-
306
- ### Implementation in current codebase
307
-
308
- 1. Add a data ingestion script under `scripts/`:
309
- - `scripts/build_exercise_knowledge_base.py`
310
- 2. Store normalized cards under:
311
- - `data/knowledge/exercises.json`
312
- 3. Update `src/pozify/knowledge_cards.py` to:
313
- - load built-in cards,
314
- - merge external exercise cards,
315
- - keep deterministic retrieval for current exercise/variation/issue labels.
316
- 4. Keep the current coach-summary flow intact.
317
-
318
- ### Benefits
319
-
320
- - Immediate product improvement.
321
- - No retraining required.
322
- - Low safety risk.
323
- - Reuses the current prompt architecture.
324
-
325
- ## Option B: Pozify-native SFT for coach summary
326
-
327
- ### Why this is the right SFT target
328
-
329
- If you do SFT, train on the task Pozify actually runs:
330
-
331
- - structured evidence in
332
- - structured grounded summary out
333
-
334
- ### Recommended data source for SFT
335
-
336
- Build your own dataset from:
337
-
338
- - current artifact pipeline outputs in `runs/`
339
- - synthetic or hand-authored expert summaries
340
- - optional edits by the team
341
-
342
- Then optionally augment with:
343
-
344
- - filtered examples from `HazSylvia/Fitness_Unformatted`
345
- - filtered examples from `onurSakar/GYM-Exercise`
346
- - filtered examples from `chibbss/fitness-chat-prompt-completion-dataset`
347
-
348
- But use public datasets only for:
349
-
350
- - tone,
351
- - phrasing,
352
- - concise coaching style,
353
- - and generic structure.
354
-
355
- Do not use them to replace the grounded target format.
356
-
357
- ### SFT training objective
358
-
359
- Input:
360
-
361
- - current prompt evidence object
362
- - retrieved cards
363
-
364
- Output:
365
-
366
- - the exact `CoachSummary` JSON contract
367
-
368
- ### Recommended model
369
-
370
- Use the current runtime-aligned model family first:
371
-
372
- - `Qwen/Qwen3-14B`
373
-
374
- Because the app already uses this model family successfully through Hugging Face Inference.
375
-
376
- ### Recommended training tools
377
-
378
- - TRL if you want closer Hugging Face-native control
379
- - Unsloth if you want fast LoRA iteration on a smaller GPU budget
380
- - Axolotl if you want a cleaner config-driven fine-tuning pipeline
381
-
382
- ### Best fine-tuning style
383
-
384
- - LoRA / QLoRA
385
- - JSON-only target format
386
- - deterministic eval checks against verifier outcomes
387
-
388
- ### Why not train directly on public chat sets first
389
-
390
- Because the task mismatch is too large:
391
-
392
- - public chat sets teach “fitness assistant”
393
- - Pozify needs “artifact-grounded explanation generator”
394
-
395
- ## Option C: Separate chat / Ask-a-Coach feature
396
-
397
- This is where the public datasets are actually strongest.
398
-
399
- ### Best datasets for that feature
400
-
401
- - `HazSylvia/Fitness_Unformatted`
402
- - `onurSakar/GYM-Exercise`
403
- - `chibbss/fitness-chat-prompt-completion-dataset`
404
-
405
- ### Suitable user stories
406
-
407
- - “I am new to the gym, where do I start?”
408
- - “Can you give me a 3-day weekly plan?”
409
- - “What should I eat after a workout?”
410
- - “How do I stay consistent?”
411
-
412
- ### Why keep this separate from coach summary
413
-
414
- Because:
415
-
416
- - the current summary step must stay grounded
417
- - broad wellness chat should not weaken form-review safety
418
- - the verifier logic is built for artifact-based outputs, not broad coaching conversations
419
-
420
- ### Suggested architecture
421
-
422
- - Keep `coach_summary` as today, evidence-first.
423
- - Add a future `ask_coach` endpoint and UI tab.
424
- - Back that feature with:
425
- - a filtered fitness Q&A dataset,
426
- - optional RAG over exercise knowledge,
427
- - and a different safety policy.
428
-
429
- ## Recommended Final Architecture
430
-
431
- ### Phase 1
432
-
433
- - Retrieval-only upgrade from exercise knowledge base
434
- - No SFT required
435
-
436
- ### Phase 2
437
-
438
- - Build Pozify-native summary dataset
439
- - Fine-tune a LoRA adapter for structured summary generation
440
-
441
- ### Phase 3
442
-
443
- - Add optional `Ask Pozify` chat mode
444
- - Use public Q&A datasets there
445
-
446
- ### Phase 4
447
-
448
- - Add personalization features using wearable/tabular data
449
- - Use `strova-ai/fitness-tracker-dataset` only if product scope expands into recommendations
450
-
451
- ## Concrete Implementation Plan
452
-
453
- ## Step 1: Add external exercise knowledge ingestion
454
-
455
- Goal:
456
-
457
- - turn exercise reference data into Pozify-ready knowledge cards
458
-
459
- Work:
460
-
461
- 1. Create `data/knowledge/` directory.
462
- 2. Add `scripts/build_exercise_knowledge_base.py`.
463
- 3. Normalize raw exercise dataset fields into a local JSON schema.
464
- 4. Add tests for normalization and card loading.
465
- 5. Update `src/pozify/knowledge_cards.py` to load external cards.
466
-
467
- Deliverables:
468
-
469
- - `data/knowledge/exercises.json`
470
- - updated knowledge card loader
471
- - tests
472
-
473
- ## Step 2: Expand current retrieval interface
474
-
475
- Goal:
476
-
477
- - use richer exercise and equipment context without changing app UX
478
-
479
- Work:
480
-
481
- 1. Keep deterministic retrieval for current labels.
482
- 2. Add equipment-aware retrieval using `UserProfile.equipment`.
483
- 3. Add goal-aware retrieval overlays.
484
- 4. Add per-exercise cue prioritization for the summary prompt.
485
-
486
- Deliverables:
487
-
488
- - richer `retrieve_cards(...)`
489
- - no changes needed in the UI contract
490
-
491
- ## Step 3: Create a Pozify-native SFT dataset builder
492
-
493
- Goal:
494
-
495
- - create training examples from the actual runtime task
496
-
497
- Work:
498
-
499
- 1. Add `scripts/build_coach_summary_sft_dataset.py`.
500
- 2. Read from:
501
- - `runs/*/exercise_classification.json`
502
- - `runs/*/rep_analysis.json`
503
- - `runs/*/variation.json`
504
- - `runs/*/issue_markers.json`
505
- - retrieved knowledge cards
506
- 3. Export JSONL rows like:
507
-
508
- ```json
509
- {
510
- "messages": [
511
- {"role": "system", "content": "..."},
512
- {"role": "user", "content": "...structured evidence..."},
513
- {"role": "assistant", "content": "{...coach summary json...}"}
514
- ]
515
- }
516
- ```
517
-
518
- 4. Start with hand-authored gold summaries for 100-300 samples.
519
- 5. Add validation scripts that reject malformed outputs.
520
-
521
- Deliverables:
522
-
523
- - `data/sft/coach_summary_train.jsonl`
524
- - `data/sft/coach_summary_eval.jsonl`
525
-
526
- ## Step 4: Use public datasets only as auxiliary style data
527
-
528
- Goal:
529
-
530
- - improve coach tone without damaging grounded reasoning
531
-
532
- Work:
533
-
534
- 1. Add `scripts/prepare_public_fitness_chat_data.py`.
535
- 2. Ingest:
536
- - `onurSakar/GYM-Exercise`
537
- - `HazSylvia/Fitness_Unformatted`
538
- - `chibbss/fitness-chat-prompt-completion-dataset`
539
- 3. Filter out:
540
- - broad medical claims
541
- - unrelated wellness topics
542
- - environment/toxin/social-life content
543
- 4. Convert only the most relevant rows into a secondary style corpus.
544
- 5. Use this corpus with lower sampling weight than Pozify-native data.
545
-
546
- Deliverables:
547
-
548
- - `data/sft/public_fitness_style.jsonl`
549
-
550
- ## Step 5: Fine-tune with LoRA
551
-
552
- Goal:
553
-
554
- - make the model better at emitting Pozify's exact summary contract
555
-
556
- Work:
557
-
558
- 1. Choose one training stack:
559
- - TRL
560
- - Unsloth
561
- - Axolotl
562
- 2. Fine-tune `Qwen/Qwen3-14B`.
563
- 3. Use:
564
- - low temperature at inference
565
- - JSON-only targets
566
- - training/eval split with verifier-based metrics
567
- 4. Track:
568
- - JSON validity rate
569
- - verifier pass rate
570
- - unsupported-issue mention rate
571
- - variation-overcorrection rate
572
-
573
- Deliverables:
574
-
575
- - LoRA adapter
576
- - eval report
577
- - deployment notes
578
-
579
- ## Step 6: Add pluggable model loading for local or Hub adapters
580
-
581
- Goal:
582
-
583
- - swap between base model and fine-tuned adapter cleanly
584
-
585
- Work:
586
-
587
- 1. Extend `src/pozify/slm/providers.py`.
588
- 2. Add env-driven model selection:
589
- - base model
590
- - fine-tuned adapter
591
- - fallback model
592
- 3. Surface model metadata in `final_report.artifacts`.
593
-
594
- Deliverables:
595
-
596
- - deployable adapter path
597
- - clear model/version tracking in UI and artifacts
598
-
599
- ## Step 7: Optional future RAG for free-form chat
600
-
601
- Goal:
602
-
603
- - add a broader coach chat feature without weakening grounded summaries
604
-
605
- Work:
606
-
607
- 1. Add `src/pozify/chat/` module.
608
- 2. Index exercise knowledge into:
609
- - FAISS first, or
610
- - ChromaDB if persistence/filtering is preferred
611
- 3. Use RAG only for:
612
- - exercise explanations
613
- - equipment substitutions
614
- - general programming suggestions
615
- 4. Keep this separate from `coach_summary`.
616
-
617
- Deliverables:
618
-
619
- - `Ask Pozify` feature
620
- - isolated inference path
621
-
622
- ## Decision Table
623
-
624
- | Dataset | Best use in Pozify | Use now? | Notes |
625
- |---|---|---:|---|
626
- | `onurSakar/GYM-Exercise` | auxiliary style SFT | No | useful tone, but too broad for summary task |
627
- | `HazSylvia/Fitness_Unformatted` | auxiliary style SFT / future chat | Later | cleaner coach Q&A, still task-mismatched |
628
- | `chibbss/fitness-chat-prompt-completion-dataset` | auxiliary style SFT / future chat | Later | small, useful for tone only |
629
- | `hasaneyldrm/exercises-dataset` | retrieval knowledge base | Yes | best immediate fit for current architecture |
630
- | `strova-ai/fitness-tracker-dataset` | future personalization | No | not aligned with current app contracts |
631
-
632
- ## Final Recommendation
633
-
634
- If the goal is to improve the current Pozify product, do this:
635
-
636
- 1. **Integrate `hasaneyldrm/exercises-dataset` as structured knowledge cards first.**
637
- 2. **Build a Pozify-native coach-summary SFT dataset from real artifacts.**
638
- 3. **Fine-tune Qwen with LoRA on the Pozify-native task.**
639
- 4. **Use public fitness chat datasets only as secondary style data.**
640
- 5. **Use RAG later for a separate chat feature, not to replace the current grounded summary path.**
641
-
642
- This path gives the best balance of:
643
-
644
- - safety,
645
- - architecture fit,
646
- - product usefulness,
647
- - and implementation risk.
648
-
649
- ## Sources
650
-
651
- - Hugging Face dataset page for `onurSakar/GYM-Exercise`: https://huggingface.co/datasets/onurSakar/GYM-Exercise
652
- - Hugging Face dataset page for `HazSylvia/Fitness_Unformatted`: https://huggingface.co/datasets/HazSylvia/Fitness_Unformatted
653
- - Hugging Face dataset page for `chibbss/fitness-chat-prompt-completion-dataset`: https://huggingface.co/datasets/chibbss/fitness-chat-prompt-completion-dataset
654
- - Hugging Face dataset page for `strova-ai/fitness-tracker-dataset`: https://huggingface.co/datasets/strova-ai/fitness-tracker-dataset
655
- - Pozify runtime code:
656
- - `src/pozify/pipeline.py`
657
- - `src/pozify/steps/coach_summary.py`
658
- - `src/pozify/slm/prompting.py`
659
- - `src/pozify/knowledge_cards.py`
660
- - `src/pozify/steps/verifier.py`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
docs/99-archive-pozify-project-documentation.md DELETED
@@ -1,1211 +0,0 @@
1
- # Pozify Project Documentation
2
-
3
- Status: large historical product/design document.
4
-
5
- Some sections may lag behind the current runtime implementation. For the current project entrypoint,
6
- commands, and model workflow, use [../README.md](../README.md) and [01-docs-index.md](01-docs-index.md).
7
-
8
- ## 1. Product Overview
9
-
10
- **Pozify** is a Gradio web app that lets a user upload or record a short workout video, then returns a rep-by-rep movement report built from pose landmarks, movement metrics, frame-level issue markers, and goal-aware coaching feedback.
11
-
12
- One-line pitch:
13
-
14
- > Pozify turns a short workout video into a rep-by-rep movement report: what you did, where your form changed, why it matters for your goal, and what to practice next.
15
-
16
- Pozify is not positioned as a generic fitness chatbot. It is a **video evidence review cockpit**: the user sees the exercise detected, rep count, issue timestamps, movement metrics, confidence notes, and a practical next-session plan.
17
-
18
- Pozify is also not a medical or clinical product. It should not diagnose injuries, claim to prevent injuries, or replace a qualified trainer, clinician, or physical therapist.
19
-
20
- ## 2. MVP Scope
21
-
22
- The MVP supports three exercises:
23
-
24
- - Squat.
25
- - Push-up.
26
- - Shoulder press.
27
-
28
- Depth strategy:
29
-
30
- - Classify all three supported exercises.
31
- - Provide deeper analysis for squat and push-up first.
32
- - Provide shoulder press counting and one or two core issue checks in the MVP.
33
- - Clearly display confidence and feature depth in the UI.
34
-
35
- ### In Scope
36
-
37
- - Gradio web app.
38
- - Upload or record short workout videos.
39
- - User profile and training intent form.
40
- - Video quality check.
41
- - 17-point 3D pose extraction.
42
- - Pose cleaning and normalization.
43
- - Exercise classifier for `squat`, `push_up`, `shoulder_press`, and `unknown`.
44
- - Exercise-specific rep counter.
45
- - Per-rep movement metrics.
46
- - Basic variation detection.
47
- - Frame-level issue markers.
48
- - Annotated video renderer.
49
- - Grounded coach summary from a small language model.
50
- - Verifier for coach summary quality and safety.
51
- - JSON evidence/debug output.
52
- - Optional downloadable JSON/PDF report.
53
-
54
- ### Out Of Scope
55
-
56
- - Medical diagnosis.
57
- - Injury claims.
58
- - Lab-grade biomechanics claims.
59
- - Support for every gym exercise.
60
- - Real-time live coaching.
61
- - Multi-person workout analysis.
62
- - Replacing a professional coach or clinician.
63
-
64
- ## 3. Target User And Problem
65
-
66
- Target users:
67
-
68
- - Beginner or casual gym-goers.
69
- - People training at home.
70
- - Users who already record workout videos but do not know how to evaluate them.
71
- - Users who need specific, timestamped feedback instead of generic advice.
72
-
73
- Core user problem:
74
-
75
- - The user has a video but does not know which reps changed or failed.
76
- - Online form advice is often generic and not tied to visible evidence.
77
- - Valid exercise variations can be mistaken for errors.
78
- - Beginners need concise cues and a concrete next action.
79
-
80
- ## 4. System Architecture
81
-
82
- Main pipeline:
83
-
84
- ```text
85
- user profile + input video
86
- -> video QC
87
- -> 17-point 3D pose landmarker
88
- -> pose cleaning and normalization
89
- -> exercise classifier
90
- -> exercise-specific rep counter
91
- -> per-rep analysis
92
- -> variation detection
93
- -> frame-level issue markers
94
- -> annotated video renderer
95
- -> grounded coach summary
96
- -> verifier
97
- -> final report and UI
98
- ```
99
-
100
- End-to-end flow:
101
-
102
- ```mermaid
103
- flowchart TD
104
- A["User uploads or records workout video"] --> B["Video QC"]
105
- U["User profile and training intent"] --> G["Grounded coach summary"]
106
- B --> C["17-point 3D Pose Landmarker"]
107
- C --> D["Pose cleaning and normalization"]
108
- D --> E["Exercise classifier"]
109
- E --> F{"Exercise type"}
110
- F -->|Squat| S1["Squat rep counter"]
111
- F -->|Push-up| P1["Push-up rep counter"]
112
- F -->|Shoulder press| O1["Shoulder press rep counter"]
113
- S1 --> S2["Squat per-rep analysis"]
114
- P1 --> P2["Push-up per-rep analysis"]
115
- O1 --> O2["Shoulder press per-rep analysis"]
116
- S2 --> H["Frame-level issue markers"]
117
- P2 --> H
118
- O2 --> H
119
- H --> R["Annotated video renderer"]
120
- H --> G
121
- K["Exercise knowledge cards"] --> G
122
- G --> V["Summary verifier"]
123
- V --> Z["Final report and UI"]
124
- R --> Z
125
- ```
126
-
127
- Data contract flow:
128
-
129
- ```mermaid
130
- flowchart LR
131
- A["input.mp4"] --> B["video_manifest.json"]
132
- B --> C["pose_sequence.json"]
133
- C --> D["exercise_classification.json"]
134
- D --> E["reps.json"]
135
- E --> F["rep_analysis.json"]
136
- F --> G["variation.json"]
137
- G --> H["issue_markers.json"]
138
- H --> I["coach_summary.json"]
139
- H --> J["annotated_video.mp4"]
140
- I --> K["final_report.json"]
141
- J --> K
142
- ```
143
-
144
- Design principles:
145
-
146
- - Each step reads and writes structured data.
147
- - Each model can be replaced independently.
148
- - Every coach summary claim should be traceable to metrics, issue markers, user profile, or knowledge cards.
149
- - Low-confidence input should produce warnings or rejection instead of hallucinated feedback.
150
-
151
- ## 5. Data Contracts
152
-
153
- The Python contracts are defined in `src/pozify/contracts.py`.
154
-
155
- ### `UserProfile`
156
-
157
- Captures user intent and context.
158
-
159
- Fields:
160
-
161
- - `goal`: `strength`, `hypertrophy`, `endurance`, `mobility`, or `beginner_practice`.
162
- - `experience_level`: `beginner` or `intermediate`.
163
- - `intended_exercise`: `auto`, `squat`, `push_up`, or `shoulder_press`.
164
- - `intended_variation`: optional variation label.
165
- - `known_limitations`: discomfort or limitation flags.
166
- - `equipment`: `bodyweight`, `dumbbell`, `barbell`, or `unknown`.
167
-
168
- Example:
169
-
170
- ```json
171
- {
172
- "goal": "hypertrophy",
173
- "experience_level": "beginner",
174
- "intended_exercise": "push_up",
175
- "intended_variation": "wide_grip_push_up",
176
- "known_limitations": ["wrist_discomfort"],
177
- "equipment": "bodyweight"
178
- }
179
- ```
180
-
181
- ### `VideoManifest`
182
-
183
- Stores video metadata and quality warnings.
184
-
185
- Example:
186
-
187
- ```json
188
- {
189
- "video_path": "input.mp4",
190
- "fps": 30.0,
191
- "duration_sec": 24.8,
192
- "total_frames": 744,
193
- "sampled_frames": 372,
194
- "quality_warnings": ["side_view_detected"],
195
- "analysis_allowed": true
196
- }
197
- ```
198
-
199
- ### `PoseSequence`
200
-
201
- Stores pose frames after extraction and cleaning.
202
-
203
- Each frame includes:
204
-
205
- - `frame_index`
206
- - `timestamp_sec`
207
- - `landmarks`
208
- - `world_landmarks`
209
- - `pose_quality`
210
-
211
- Example:
212
-
213
- ```json
214
- {
215
- "frame_index": 120,
216
- "timestamp_sec": 4.0,
217
- "landmarks": {
218
- "left_shoulder": { "x": 0.43, "y": 0.32, "z": -0.03, "visibility": 0.99 }
219
- },
220
- "world_landmarks": {},
221
- "pose_quality": {
222
- "mean_visibility": 0.93,
223
- "critical_landmarks_visible": true
224
- }
225
- }
226
- ```
227
-
228
- ### `ExerciseClassification`
229
-
230
- Routes the video to the correct analyzer.
231
-
232
- Example:
233
-
234
- ```json
235
- {
236
- "exercise": "push_up",
237
- "confidence": 0.92,
238
- "window_predictions": [
239
- { "start_sec": 0.0, "end_sec": 1.0, "label": "push_up", "confidence": 0.91 }
240
- ],
241
- "fallback_required": false
242
- }
243
- ```
244
-
245
- ### `Reps`
246
-
247
- Stores rep boundaries.
248
-
249
- Example:
250
-
251
- ```json
252
- {
253
- "exercise": "push_up",
254
- "reps": [
255
- {
256
- "rep_id": 1,
257
- "start_frame": 60,
258
- "mid_frame": 91,
259
- "end_frame": 125,
260
- "start_sec": 2.0,
261
- "mid_sec": 3.03,
262
- "end_sec": 4.17
263
- }
264
- ],
265
- "partial_reps": []
266
- }
267
- ```
268
-
269
- ### `RepAnalysis`
270
-
271
- Stores per-rep metrics and aggregate metrics.
272
-
273
- Example:
274
-
275
- ```json
276
- {
277
- "rep_id": 4,
278
- "duration_sec": 1.8,
279
- "range_of_motion_score": 0.74,
280
- "stability_score": 0.66,
281
- "symmetry_score": 0.82,
282
- "metrics": {
283
- "min_elbow_angle_deg": 91,
284
- "body_line_score": 0.62,
285
- "hip_sag_score": 0.78,
286
- "hand_width_ratio": 1.42
287
- },
288
- "variation_hints": ["wide_grip_push_up"]
289
- }
290
- ```
291
-
292
- ### `Variation`
293
-
294
- Separates valid exercise variation from true issues.
295
-
296
- Example:
297
-
298
- ```json
299
- {
300
- "exercise": "push_up",
301
- "detected_variation": "wide_grip_push_up",
302
- "variation_confidence": 0.84,
303
- "not_issues": ["wide_hand_placement"]
304
- }
305
- ```
306
-
307
- ### `IssueMarkers`
308
-
309
- Stores issue intervals and evidence.
310
-
311
- Example:
312
-
313
- ```json
314
- {
315
- "issues": [
316
- {
317
- "rep_id": 4,
318
- "issue": "hip_sag",
319
- "severity": 0.78,
320
- "start_frame": 210,
321
- "end_frame": 248,
322
- "start_sec": 7.0,
323
- "end_sec": 8.27,
324
- "affected_joints": ["left_hip", "right_hip", "shoulders", "ankles"],
325
- "evidence": {
326
- "body_line_score": 0.52,
327
- "threshold": 0.65
328
- }
329
- }
330
- ]
331
- }
332
- ```
333
-
334
- ### `CoachSummary`
335
-
336
- Stores grounded user-facing feedback.
337
-
338
- Fields:
339
-
340
- - `summary`
341
- - `what_went_well`
342
- - `main_findings`
343
- - `variation_explanation`
344
- - `top_fixes`
345
- - `next_session_plan`
346
- - `confidence_notes`
347
-
348
- ### `Verification`
349
-
350
- Stores summary verification results.
351
-
352
- Fields:
353
-
354
- - `passed`
355
- - `checks`
356
- - `notes`
357
-
358
- ## 6. Step-By-Step Implementation Plan
359
-
360
- ### Step 0: User Profile And Training Intent
361
-
362
- Goal:
363
-
364
- - Collect context so coaching is not one-size-fits-all.
365
- - Help distinguish valid variations from issues.
366
- - Adapt feedback to the user's goal and experience level.
367
-
368
- Model:
369
-
370
- - No model required.
371
-
372
- Current implementation:
373
-
374
- - Gradio form values are converted into a `UserProfile` contract.
375
-
376
- Future implementation:
377
-
378
- - Add stricter validation for unsupported exercise/variation combinations.
379
- - Add presets for beginner, hypertrophy, strength, endurance, and mobility use cases.
380
-
381
- ### Step 1: Video Intake And Quality Check
382
-
383
- Goal:
384
-
385
- - Accept the video.
386
- - Normalize metadata.
387
- - Decide whether analysis is allowed.
388
-
389
- Model:
390
-
391
- - No dedicated model required.
392
-
393
- Future algorithm:
394
-
395
- 1. Decode video with OpenCV.
396
- 2. Limit duration to 10-60 seconds.
397
- 3. Resize to 480p or 720p.
398
- 4. Sample at 15-30 FPS.
399
- 5. Check brightness.
400
- 6. Check blur with variance of Laplacian.
401
- 7. Check frame count and FPS.
402
- 8. Check pose-valid ratio after pose detection.
403
- 9. Check full-body visibility.
404
- 10. Warn or reject if there are multiple people in frame.
405
-
406
- Current implementation:
407
-
408
- - Returns mocked video metadata and warnings.
409
-
410
- ### Step 2: 17-Point 3D Pose Landmarker
411
-
412
- Goal:
413
-
414
- - Convert video frames into full-body pose landmarks.
415
-
416
- Recommended model:
417
-
418
- - MediaPipe Pose Landmarker.
419
-
420
- Reasons:
421
-
422
- - Works on decoded video frames.
423
- - Outputs COCO-17 body landmarks.
424
- - Provides image coordinates for rendering and world coordinates for 3D exercise metrics.
425
- - Practical for a Gradio Space.
426
-
427
- Fine-tuning:
428
-
429
- - Do not fine-tune the pose detector for the MVP.
430
- - Use it as a stable feature extractor.
431
-
432
- Current implementation:
433
-
434
- - Generates mocked landmark frames.
435
-
436
- Future implementation:
437
-
438
- - Run MediaPipe on sampled frames.
439
- - Store all 17 COCO body landmarks.
440
- - Store visibility, pose quality, and 3D world coordinates per frame.
441
-
442
- ### Step 3: Pose Cleaning And Normalization
443
-
444
- Goal:
445
-
446
- - Make landmark sequences stable enough for classification and metrics.
447
-
448
- Future algorithm:
449
-
450
- - Drop low-confidence frames.
451
- - Interpolate short missing spans.
452
- - Smooth landmarks with Savitzky-Golay, exponential smoothing, or OneEuro filter.
453
- - Normalize coordinates by hip center and torso length.
454
- - Optionally normalize camera rotation for front/side views.
455
-
456
- Current implementation:
457
-
458
- - Marks the mocked sequence as normalized and smoothed.
459
-
460
- ### Step 4: Exercise Classifier
461
-
462
- Goal:
463
-
464
- - Classify video as `squat`, `push_up`, `shoulder_press`, or `unknown`.
465
-
466
- Recommended model:
467
-
468
- - BiLSTM, TCN, tiny 1D CNN, or GRU over pose-feature windows.
469
-
470
- Fast baseline:
471
-
472
- - XGBoost or LightGBM over engineered sequence features.
473
-
474
- Fine-tuning:
475
-
476
- - This is the first recommended fine-tune.
477
- - It is easier to evaluate and explain than a freeform language-model fine-tune.
478
-
479
- Input features:
480
-
481
- - Normalized landmarks.
482
- - Joint angles: knee, hip, elbow, shoulder.
483
- - Relative distances:
484
- - hand width / shoulder width.
485
- - stance width / hip width.
486
- - wrist-to-shoulder.
487
- - hip-to-knee.
488
- - Motion deltas:
489
- - wrist velocity.
490
- - hip velocity.
491
- - knee velocity.
492
- - joint angle deltas.
493
-
494
- Current implementation:
495
-
496
- - Uses the intended exercise if provided.
497
- - Defaults to mocked `push_up` when `auto` is selected.
498
-
499
- ### Step 5: Rep Counting
500
-
501
- Goal:
502
-
503
- - Split the movement into individual reps.
504
- - Return `start`, `mid`, and `end` frames and timestamps.
505
-
506
- Model:
507
-
508
- - No ML needed for MVP.
509
- - Use deterministic signal processing and exercise-specific state machines.
510
-
511
- Squat:
512
-
513
- - Signal: knee angle and hip vertical position.
514
- - Rep: top -> bottom -> top.
515
-
516
- Push-up:
517
-
518
- - Signal: elbow angle, shoulder/chest vertical motion, and body line.
519
- - Rep: top -> bottom -> top.
520
-
521
- Shoulder press:
522
-
523
- - Signal: wrist height and elbow angle.
524
- - Rep: bottom -> top -> bottom.
525
-
526
- Current implementation:
527
-
528
- - Returns five mocked reps.
529
-
530
- ### Step 6: Per-Rep Analysis
531
-
532
- Goal:
533
-
534
- - Produce a rich feature pack for each rep.
535
- - Avoid reducing form review to a generic good/bad label.
536
-
537
- Common metrics:
538
-
539
- - Rep duration.
540
- - Eccentric duration.
541
- - Concentric duration.
542
- - Pause at top/bottom.
543
- - Tempo consistency.
544
- - Range of motion score.
545
- - Left/right symmetry score.
546
- - Stability score.
547
- - Smoothness/jerk score.
548
- - Landmark confidence.
549
- - Fatigue trend across reps.
550
-
551
- Squat metrics:
552
-
553
- - Min/max knee angle.
554
- - Min/max hip angle.
555
- - Hip depth relative to knee.
556
- - Torso lean.
557
- - Knee valgus/varus.
558
- - Stance width.
559
- - Hip shift.
560
- - Bottom stability.
561
-
562
- Push-up metrics:
563
-
564
- - Min/max elbow angle.
565
- - Body line score.
566
- - Hip sag or pike score.
567
- - Chest depth proxy.
568
- - Hand width ratio.
569
- - Elbow flare.
570
- - Neck/head alignment.
571
- - Lockout quality.
572
-
573
- Shoulder press metrics:
574
-
575
- - Min/max elbow angle.
576
- - Wrist path verticality.
577
- - Lockout quality.
578
- - Left/right wrist height asymmetry.
579
- - Back arch proxy.
580
- - Overhead stability.
581
-
582
- Current implementation:
583
-
584
- - Returns exercise-aware mocked metrics.
585
-
586
- ### Step 7: Variation Detection
587
-
588
- Goal:
589
-
590
- - Separate valid variation from true issue.
591
-
592
- Examples:
593
-
594
- - Wide-grip push-up is not automatically an error.
595
- - Knee push-up can be a regression, not a failure.
596
- - Partial squat may be acceptable depending on intent.
597
-
598
- Model:
599
-
600
- - Rule-based first.
601
- - Optional classifier later.
602
-
603
- Current implementation:
604
-
605
- - Returns a mocked variation, or the user's intended variation if provided.
606
-
607
- ### Step 8: Frame-Level Issue Markers
608
-
609
- Goal:
610
-
611
- - Identify exact frames/timestamps where issues occur.
612
- - Attach issues to reps.
613
- - Provide evidence for annotated video and coach summary.
614
-
615
- Model:
616
-
617
- - Hybrid:
618
- - Rule-based issue scores for transparency.
619
- - Optional fine-tuned temporal classifier for robustness.
620
-
621
- Initial issue labels:
622
-
623
- - `shallow_depth`
624
- - `knee_valgus`
625
- - `excessive_torso_lean`
626
- - `hip_shift`
627
- - `hip_sag`
628
- - `incomplete_depth`
629
- - `incomplete_lockout`
630
- - `uneven_tempo`
631
- - `unstable_bottom`
632
- - `asymmetry`
633
-
634
- Current implementation:
635
-
636
- - Creates mocked issue intervals when stability drops below a threshold.
637
-
638
- ### Step 9: Annotated Video Renderer
639
-
640
- Goal:
641
-
642
- - Turn movement analysis into a visual review.
643
-
644
- Future renderer:
645
-
646
- - Draw skeleton on every frame.
647
- - Draw normal skeleton in green or blue.
648
- - Draw active issue joints/segments in red or orange.
649
- - Overlay rep count.
650
- - Overlay current phase.
651
- - Overlay issue label during issue intervals.
652
- - Add confidence warnings when needed.
653
- - Export `annotated_video.mp4`.
654
-
655
- Current implementation:
656
-
657
- - Writes `annotated_video_placeholder.json`.
658
- - Returns the original video path.
659
-
660
- ### Step 10: Grounded Coach Summary
661
-
662
- Goal:
663
-
664
- - Convert structured evidence into concise PT-style feedback.
665
- - Avoid model hallucinations.
666
- - Separate valid variations from actual issues.
667
-
668
- Recommended SLM:
669
-
670
- - Qwen2.5-3B-Instruct for multilingual and structured output.
671
- - SmolLM2-1.7B-Instruct for a smaller model story.
672
- - MiniCPM4-0.5B if the team wants an aggressive tiny-model angle.
673
-
674
- The SLM should receive:
675
-
676
- - User profile.
677
- - Exercise classification.
678
- - Variation detection.
679
- - Rep analysis.
680
- - Issue markers.
681
- - Retrieved exercise knowledge cards.
682
-
683
- The SLM should not:
684
-
685
- - Detect issues directly from video.
686
- - Invent metrics.
687
- - Diagnose injury.
688
- - Claim injury prevention.
689
- - Treat valid variation as an error.
690
-
691
- Current implementation:
692
-
693
- - Returns a mocked summary based on structured artifacts.
694
-
695
- ### Step 11: Verifier
696
-
697
- Goal:
698
-
699
- - Check the coach summary before displaying it.
700
-
701
- Verifier checks:
702
-
703
- - Does the summary mention only issues present in JSON?
704
- - Does it separate variation from issue?
705
- - Does it adapt to user goal and experience?
706
- - Does it avoid medical diagnosis?
707
- - Does it include confidence notes when needed?
708
-
709
- Current implementation:
710
-
711
- - Runs a lightweight mocked rule verifier.
712
-
713
- ## 7. Model Plan
714
-
715
- ### Pose Detector
716
-
717
- Recommended model:
718
-
719
- - MediaPipe Pose Landmarker.
720
-
721
- Role:
722
-
723
- - Extract 17 COCO body landmarks per valid frame, with 3D world coordinates when available.
724
-
725
- Fine-tuning:
726
-
727
- - No fine-tuning for MVP.
728
-
729
- ### Exercise Router
730
-
731
- Recommended model:
732
-
733
- - BiLSTM, TCN, tiny 1D CNN, or GRU over pose windows.
734
-
735
- Role:
736
-
737
- - Route to the correct exercise-specific analyzer.
738
-
739
- Fine-tuning:
740
-
741
- - Yes. This is the highest-priority fine-tune.
742
-
743
- ### Rep Counter
744
-
745
- Model:
746
-
747
- - No ML for MVP.
748
-
749
- Role:
750
-
751
- - Segment reps using deterministic state machines.
752
-
753
- ### Per-Rep Analyzer
754
-
755
- Model:
756
-
757
- - Rule-based metrics and biomechanical feature engineering.
758
-
759
- Role:
760
-
761
- - Produce evidence for issue markers and coach summary.
762
-
763
- ### Issue Classifier
764
-
765
- Recommended model:
766
-
767
- - Tiny 1D CNN, TCN, GRU, XGBoost, or LightGBM.
768
-
769
- Role:
770
-
771
- - Predict issue probabilities per rep or frame interval.
772
-
773
- Fine-tuning:
774
-
775
- - Optional after rule-based markers work.
776
-
777
- ### Small Language Model
778
-
779
- Recommended models:
780
-
781
- - Qwen2.5-3B-Instruct.
782
- - SmolLM2-1.7B-Instruct.
783
- - MiniCPM4-0.5B.
784
-
785
- Role:
786
-
787
- - Explain structured findings in clear language.
788
- - Generate top fixes and next-session plan.
789
-
790
- Fine-tuning:
791
-
792
- - Optional LoRA/SFT for tone, format, and grounded reasoning.
793
- - Do not use the SLM fine-tune as the primary source of fitness knowledge.
794
-
795
- ## 8. Dataset Plan
796
-
797
- ### Exercise Router Dataset
798
-
799
- Primary:
800
-
801
- - `RickyRiccio/Real_Time_Exercise_Recognition_Dataset`
802
-
803
- Useful labels:
804
-
805
- - Squat.
806
- - Push-up.
807
- - Shoulder press.
808
- - Barbell bicep curl as a possible negative class.
809
-
810
- Custom additions:
811
-
812
- - Unknown/setup clips.
813
- - Standing idle.
814
- - Walking into or out of frame.
815
- - Stretching.
816
- - Bad camera angles.
817
- - Partial reps.
818
- - Team/friend videos to improve robustness.
819
-
820
- ### Issue Classifier Dataset
821
-
822
- Public or academic options:
823
-
824
- - Fitness-AQA:
825
- - BackSquat, OverheadPress, and BarbellRow.
826
- - Posture-error annotations.
827
- - Requires access approval and attention to non-commercial terms.
828
- - FLEX:
829
- - 20 fitness actions.
830
- - Multiview video.
831
- - 3D pose.
832
- - Error types and feedback.
833
- - Requires terms review.
834
-
835
- Practical hackathon dataset:
836
-
837
- - Record 50-100 clips.
838
- - Label per rep:
839
- - good.
840
- - shallow depth.
841
- - hip sag.
842
- - incomplete lockout.
843
- - torso lean.
844
- - unstable bottom.
845
- - knee valgus.
846
- - asymmetry.
847
-
848
- Labeling strategy:
849
-
850
- 1. Generate pseudo-labels from rules.
851
- 2. Manually correct a small validation set.
852
- 3. Train a tiny classifier.
853
- 4. Compare classifier output with rule baseline.
854
-
855
- ### Coach Summary SFT Dataset
856
-
857
- Goal:
858
-
859
- - Fine-tune style and structure, not factual knowledge.
860
-
861
- Training format:
862
-
863
- ```json
864
- {
865
- "analysis_json": {},
866
- "user_profile": {},
867
- "retrieved_knowledge_cards": [],
868
- "ideal_coach_summary": ""
869
- }
870
- ```
871
-
872
- Target size:
873
-
874
- - 100-300 examples.
875
-
876
- Sources:
877
-
878
- - Synthetic rep analysis JSON.
879
- - Curated exercise cards.
880
- - Team-written ideal summaries.
881
- - Ambiguous cases:
882
- - wide-grip push-up.
883
- - knee push-up.
884
- - partial squat.
885
- - shoulder press back arch.
886
- - low-confidence camera angle.
887
-
888
- ## 9. Knowledge Cards
889
-
890
- Knowledge cards keep the SLM grounded.
891
-
892
- Card types:
893
-
894
- - Exercise card.
895
- - Variation card.
896
- - Issue card.
897
- - Goal card.
898
- - Safety card.
899
-
900
- Each card should contain:
901
-
902
- - Exercise or variation label.
903
- - Primary muscles.
904
- - Secondary muscles.
905
- - Valid form notes.
906
- - Common issues.
907
- - Related metrics.
908
- - Not-always-errors.
909
- - Coaching cues.
910
- - Regressions.
911
- - Progressions.
912
- - Safety notes.
913
- - Claims the assistant must not make.
914
-
915
- Example:
916
-
917
- ```json
918
- {
919
- "exercise": "push_up",
920
- "variation": "wide_grip_push_up",
921
- "primary_muscles": ["chest"],
922
- "secondary_muscles": ["triceps", "front_delts", "core"],
923
- "valid_form_notes": [
924
- "Hands wider than shoulder width",
925
- "More chest emphasis than close-grip push-up",
926
- "Some elbow flare is expected"
927
- ],
928
- "common_issues": ["hip_sag", "incomplete_depth"],
929
- "not_always_errors": ["wide_hand_placement", "moderate_elbow_flare"],
930
- "coaching_cues": [
931
- "Brace before descent",
932
- "Keep shoulders and hips moving together"
933
- ]
934
- }
935
- ```
936
-
937
- Retrieval:
938
-
939
- - MVP: deterministic lookup by labels.
940
- - Later: small embedding model if card count grows.
941
-
942
- ## 10. Output And UI Strategy
943
-
944
- Final user-facing output:
945
-
946
- 1. Annotated video with skeleton and issue markers.
947
- 2. Exercise type and confidence.
948
- 3. Detected variation.
949
- 4. Rep count.
950
- 5. Overall form score.
951
- 6. Movement metrics.
952
- 7. Per-rep breakdown.
953
- 8. Issue timeline with timestamps.
954
- 9. PT-style explanation.
955
- 10. Next-session fix plan.
956
- 11. JSON/PDF report if available.
957
-
958
- ### Scan Summary
959
-
960
- Show immediately:
961
-
962
- - Detected exercise.
963
- - Detected variation.
964
- - Completed reps.
965
- - Overall form score.
966
- - Main issue.
967
- - Biggest win.
968
- - Confidence.
969
-
970
- Example:
971
-
972
- ```text
973
- Exercise: Push-up
974
- Variation: Wide-grip
975
- Reps: 8
976
- Overall Form: 74/100
977
- Main Limiter: Hip sag after rep 5
978
- Confidence: 87%
979
- ```
980
-
981
- ### Annotated Video
982
-
983
- Primary visual:
984
-
985
- - Video player with skeleton overlay.
986
- - Red affected joints only during issue intervals.
987
- - Rep counter.
988
- - Timeline markers.
989
- - Click-to-jump issue cards.
990
-
991
- ### Movement Metrics
992
-
993
- Each metric should include:
994
-
995
- - Value.
996
- - Target or expected band.
997
- - Status: good, caution, needs work.
998
- - Short explanation.
999
-
1000
- ### Rep Breakdown
1001
-
1002
- Each rep should include:
1003
-
1004
- - Worst-frame thumbnail.
1005
- - ROM score.
1006
- - Stability score.
1007
- - Symmetry score.
1008
- - Tempo.
1009
- - Issues.
1010
- - Confidence.
1011
-
1012
- ### Fix Plan
1013
-
1014
- The report should end with practical next actions, not only a summary.
1015
-
1016
- Example:
1017
-
1018
- ```text
1019
- Next session:
1020
- 1. Slow negative push-up - 2 sets x 5 reps
1021
- Focus: keep shoulders, hips, and ankles moving as one line.
1022
-
1023
- 2. High plank hold - 2 x 20s
1024
- Focus: brace before each rep.
1025
-
1026
- 3. Stop set when hip sag appears
1027
- Your form starts breaking down after rep 5 today.
1028
- ```
1029
-
1030
- ### Debug Evidence
1031
-
1032
- For hackathon credibility, expose:
1033
-
1034
- - JSON debug tab.
1035
- - Classifier confidence.
1036
- - Landmark quality.
1037
- - Issue thresholds.
1038
- - Window predictions.
1039
- - Export report.
1040
-
1041
- ## 11. Safety And Risk Guardrails
1042
-
1043
- ### Risk: The App Feels Generic
1044
-
1045
- Mitigation:
1046
-
1047
- - Do not position Pozify as a chatbot.
1048
- - Make frame-level markers the hero.
1049
- - Show classifier confidence, rep segmentation, issue intervals, and metrics.
1050
-
1051
- ### Risk: Bad Video Quality Breaks Analysis
1052
-
1053
- Mitigation:
1054
-
1055
- - Add capture guidance.
1056
- - Add a video quality gate.
1057
- - Refuse low-quality analysis.
1058
- - Show confidence and camera warnings.
1059
-
1060
- ### Risk: Three Exercises Become Too Shallow
1061
-
1062
- Mitigation:
1063
-
1064
- - Classify all three.
1065
- - Analyze squat and push-up deeply first.
1066
- - Keep shoulder press simpler in the MVP.
1067
-
1068
- ### Risk: Coach Summary Is Wrong
1069
-
1070
- Mitigation:
1071
-
1072
- - Feed structured JSON, knowledge cards, and user profile.
1073
- - Add a verifier step.
1074
- - Keep variation labels separate from issue labels.
1075
-
1076
- ### Risk: Fitness Safety Claims
1077
-
1078
- Mitigation:
1079
-
1080
- - Use "practice feedback" language.
1081
- - Avoid diagnosis and injury-prevention claims.
1082
- - Mention uncertainty when confidence is low.
1083
- - If discomfort is reported, give conservative advice.
1084
-
1085
- Recommended language:
1086
-
1087
- - "This is practice feedback based on visible movement."
1088
- - "If this causes pain or discomfort, stop and ask a qualified professional."
1089
-
1090
- Avoid:
1091
-
1092
- - "This prevents injury."
1093
- - "You have a knee problem."
1094
- - "This is a medical assessment."
1095
-
1096
- ## 12. Build Phases
1097
-
1098
- ### Phase 1: Core Pipeline
1099
-
1100
- Deliver:
1101
-
1102
- - Gradio UI.
1103
- - Video upload.
1104
- - OpenCV decode.
1105
- - Basic video QC.
1106
- - MediaPipe pose extraction.
1107
- - Pose JSON.
1108
- - Basic annotated video export.
1109
-
1110
- ### Phase 2: Exercise And Reps
1111
-
1112
- Deliver:
1113
-
1114
- - Exercise classifier.
1115
- - Rep counters for three exercises.
1116
- - Confidence and manual fallback.
1117
- - Rep boundary visualization.
1118
-
1119
- ### Phase 3: Analysis And Markers
1120
-
1121
- Deliver:
1122
-
1123
- - Per-rep metrics.
1124
- - Variation detection.
1125
- - Frame-level issue markers.
1126
- - Issue timeline.
1127
- - Rep review table.
1128
-
1129
- ### Phase 4: Coach Summary
1130
-
1131
- Deliver:
1132
-
1133
- - Knowledge cards.
1134
- - Deterministic retrieval.
1135
- - SLM summary.
1136
- - Fixed JSON output format.
1137
- - Verifier.
1138
- - Conservative fallback summary.
1139
-
1140
- ### Phase 5: Polish
1141
-
1142
- Deliver:
1143
-
1144
- - Better Gradio layout.
1145
- - Before/after demo.
1146
- - Export report.
1147
- - JSON debug tab.
1148
- - Optional progress comparison view.
1149
-
1150
- ## 13. Hackathon Demo Script
1151
-
1152
- Use two videos:
1153
-
1154
- 1. Good-form example.
1155
- 2. Bad-form example, such as push-up hip sag after rep 5 or shallow squat depth.
1156
-
1157
- Demo flow:
1158
-
1159
- 1. Upload video.
1160
- 2. App detects `push_up`.
1161
- 3. App detects `wide_grip_push_up`.
1162
- 4. App counts 8 reps.
1163
- 5. Timeline shows issues in reps 6-8.
1164
- 6. User clicks marker and the video jumps to the issue frame.
1165
- 7. Metrics show body line score dropping after rep 5.
1166
- 8. Coach summary explains:
1167
- - Wide grip is not the issue.
1168
- - Hip sag is the actual breakdown.
1169
- - The likely cause is fatigue or bracing loss.
1170
- - The next session should include 2-3 corrective drills.
1171
-
1172
- Demo one-liner:
1173
-
1174
- > Pozify turns workout video into visible movement evidence: exact reps, exact issue frames, grounded metrics, and the next practice plan.
1175
-
1176
- ## 14. Success Metrics
1177
-
1178
- Product metrics:
1179
-
1180
- - User understands what to fix in under one minute.
1181
- - Output includes concrete timestamps.
1182
- - User can see which reps were stronger or weaker.
1183
- - User receives a next-session plan.
1184
-
1185
- Technical metrics:
1186
-
1187
- - Exercise classifier works across three exercises plus unknown.
1188
- - Rep count is close on 10-60 second videos.
1189
- - Issue markers avoid one-frame noise.
1190
- - Coach summary does not mention issues outside the JSON.
1191
- - Low-quality video is warned or rejected.
1192
-
1193
- Hackathon metrics:
1194
-
1195
- - Clear visual demo.
1196
- - Defensible small-model or fine-tuned-model story.
1197
- - Dataset/model story on Hugging Face.
1198
- - JSON evidence/debug output to show the app is not a black box.
1199
-
1200
- ## 15. References
1201
-
1202
- - Build Small Hackathon: https://huggingface.co/build-small-hackathon
1203
- - MediaPipe Pose Landmarker: https://developers.google.com/edge/mediapipe/solutions/vision/pose_landmarker/python
1204
- - RickyRiccio/Real_Time_Exercise_Recognition_Dataset: https://huggingface.co/datasets/RickyRiccio/Real_Time_Exercise_Recognition_Dataset
1205
- - Fitness AI Trainer With Automatic Exercise Recognition and Counting: https://github.com/RiccardoRiccio/Fitness-AI-Trainer-With-Automatic-Exercise-Recognition-and-Counting
1206
- - Fitness-AQA: https://github.com/ParitoshParmar/Fitness-AQA
1207
- - FLEX Dataset: https://haoyin116.github.io/FLEX_Dataset/
1208
- - Qwen2.5-3B-Instruct: https://huggingface.co/Qwen/Qwen2.5-3B-Instruct
1209
- - SmolLM2-1.7B-Instruct: https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct
1210
- - Ochy: https://www.ochy.io/
1211
- - Ochy Runners: https://www.ochy.io/runners