Methodology
How we review tools, assess rights risk, run benchmarks, and maintain data accuracy.
Tool Review Process
Every tool review follows a standardized process:
- 1Account creation and testing. We create accounts on both free and paid tiers. We generate at least 20 tracks across 12 standardized genres.
- 2Terms of service review. We read the full terms of service, privacy policy, and any supplementary licensing documents. We note commercial use rights, ownership claims, and indemnification language.
- 3Rights risk assessment. We evaluate training data transparency, active litigation, commercial use clarity, and watermarking practices to assign a Low, Medium, High, or Unknown rights risk rating.
- 4Publication and review schedule. Reviews are published with a "Last Reviewed" date. We re-evaluate tools every 90 days or when significant changes occur (pricing updates, new model releases, legal developments).
Rights Risk Scoring
Rights risk is assessed on four dimensions:
Training Data Transparency
Does the company disclose what data was used to train the model? Full disclosure reduces risk. No disclosure increases it.
Active Litigation
Is the company facing copyright lawsuits? Active suits increase risk due to potential downstream liability for users.
Commercial Use Clarity
Are commercial use terms clear, unambiguous, and well-documented? Vague terms increase risk.
Watermarking and Provenance
Does the tool apply watermarks or metadata identifying content as AI-generated? Transparency in provenance reduces long-term risk.
Benchmark Testing
Audio quality benchmarks use a standardized set of 12 genre prompts tested on each model. Metrics include:
- Audio quality: Spectral analysis measuring frequency response, dynamic range, and artifact presence. Supplemented by structured listening tests.
- Genre accuracy: How well the output matches the requested genre in terms of instrumentation, rhythm, harmony, and production style.
- Vocal fidelity: Naturalness of AI-generated singing, measured by pitch accuracy, timing, and absence of artifacts.
- Generation speed: Time from prompt submission to playable output, measured across 10 generations per model.
Data Sources
Our information comes from:
- Direct testing of tools (paid accounts)
- Published terms of service and licensing agreements
- Court filings and legal analysis from music law practitioners
- Platform policy documentation and announcements
- US Copyright Office guidance and decisions
- EU AI Act documentation
Corrections
If we get something wrong, we correct it. Corrections are noted inline with the original content and dated. We do not silently edit published reviews.