Methodology
Version 0.2 — how we assess services, score dimensions, and produce recommendations.
Foundational principle
The tool exists to serve the user's mission. It is free to use, carries no advertising, and is supported entirely by voluntary contributions. Recommendations are produced by an algorithm that has no knowledge of any commercial relationship. If a recommendation could be influenced by revenue, the recommendation is wrong.
The seven dimensions
Every service is assessed against seven dimensions. You rank these in order of importance to your organisation — that ranking shapes how scores are weighted.
1. Privacy and data handling
What the service does with user data — collection, retention, sharing, monetisation, encryption practices, and surveillance exposure. This includes what data is collected (content, metadata, behavioural), how long it is retained, who it is shared with, whether it is sold or used for advertising or AI training, encryption practices in transit and at rest, end-to-end encryption availability, government access exposure, and breach history.
2. Data sovereignty and jurisdiction
Where data is physically stored, what legal regime governs it, and what that means for users in the UK and EU. This includes data centre locations, corporate jurisdiction, exposure to legal frameworks such as FISA Section 702 and the CLOUD Act, EU adequacy status, ability to legally guarantee EU residency, and transparency around government data requests.
3. Openness and interoperability
How locked-in users become, and whether the underlying technology is open. This includes open source status of the core software, support for open standards and interoperability, data portability and export, vendor lock-in factors such as proprietary formats and migration difficulty, and API openness.
4. Ownership and governance
Who owns the company, what the funding model is, and what regulatory and legal findings relate to governance. This includes ownership structure (legal form, jurisdiction, ultimate parent company), funding model (VC-backed, bootstrapped, donor-supported, member-owned), public regulatory designations and certifications, acquisition history as documented fact, final regulatory findings and court judgments, and disclosed major shareholders.
5. Environmental impact
Energy practices, emissions transparency, and renewable commitments. This includes data centre energy mix, renewable energy commitments and their verifiability, emissions reporting transparency and quality (scope 1, 2, and 3 disclosure), green hosting certifications, and hardware lifecycle considerations where applicable. This is the dimension most vulnerable to corporate greenwashing — the tool leans on third-party verification rather than corporate self-reporting.
6. Documented findings — labour
Documented findings on worker treatment, union recognition, labour enforcement, and supply chain practices, drawn from final regulatory findings, court judgments, ILO supervisory body decisions, and company-published workforce data. Coverage is sparser than other dimensions because adjudicated labour findings are genuinely rarer — the tool records what is documented and notes gaps honestly.
7. Accessibility and inclusion
Whether the service works for users with disabilities, supports assistive technology, and is inclusive in its design. This includes WCAG compliance level, accessibility statement quality, screen reader and keyboard navigation support, language coverage, and considerations for users on slow connections or older devices.
Flags and valences
Each assessed fact about a service is recorded as a flag. Every flag has a valence — a label describing whether the fact is positive, negative, neutral, or contextual.
A genuinely positive practice, clearly evidenced.
A concern worth noting but not a disqualifying factor.
A significant issue that weighs against the service.
Factual information that helps interpret other flags — neither positive nor negative.
Scoring
Valence weights
Priority weights
The dimension you rank first gets a multiplier of 7, second gets 6, down to 1 for your lowest priority. A strength in your top-ranked dimension counts far more than one in a dimension you consider least important.
Recommendation threshold
An alternative is only surfaced as a recommendation if its weighted score is at least 10 points higher than your current tool. This filters out alternatives that are only marginally better — a small gap may not justify the cost of switching.
Editorial process
Every flag in the catalogue is:
- —Grounded in sources. Every flag links to the specific source it draws on. A flag can only assert what a source actually states — silence in the evidence is not treated as evidence of absence.
- —Editorially assessed. Flags represent Open Eye Media’s honest assessment of what sources say — not legal findings or regulatory determinations. Valences (strength, caution, concern, context) are editorial labels applied to observed practices.
- —Reviewed before publication. Flags are drafted by an AI research tool and reviewed editorially before publication. The AI cannot publish directly.
- —Dated. Every flag records when it was last verified. Practices change, and the catalogue reflects that.
Source quality
Not all sources carry the same weight. The catalogue distinguishes between self-reported information and independent findings:
- —Independent sources — regulator decisions, court filings, third-party audits, investigative journalism, and academic research. These carry the most weight because they are not controlled by the subject organisation.
- —Company statements — privacy policies, terms of service, blog posts, and marketing pages. These are useful for recording what a service says about itself, but what a company says it does and what it does are not always the same. Flags based solely on company statements are noted as such.
Where a flag draws on only company-owned documentation, the editor notes this and may add an open question for future research. The goal is to over time increase the proportion of flags that are independently corroborated.
A living catalogue
This is not a static resource. The catalogue is re-researched on a rolling four-week cycle. Each cycle re-checks every service for:
- —New evidence. Privacy policies, terms, audits, and news coverage are re-checked to catch changes the service may not have announced.
- —Ownership and structure changes. Acquisitions, funding rounds, and changes to corporate structure can shift a service's alignment significantly.
- —Flag accuracy. Existing flags are re-verified against their sources. Flags whose sources no longer support the claim are updated or removed.
- —New services. New tools and better-aligned alternatives are added to the catalogue as they are identified.
Every flag carries a last-verified date. If that date is more than the expected review cadence for its category, the flag is flagged as overdue in your results. You can also submit a correction if you believe a flag is inaccurate.
What changed in v0.2
- Flag version history — every flag has a permanent permalink and a full version chain.
- Alternative-priority recommendations — when no qualifying alternative exists under stated priorities, the engine probes reweighted orderings and surfaces what would emerge.
- Descriptive score-delta labels (“Significantly better aligned”, etc.) replacing raw point deltas.
- Six-branch assessment summary sentence.
- Source rendering on result pages, with verified dates and a “verification overdue” indicator.
- Catalogue request acknowledgement for services not yet in the catalogue.
- Review cadences formalised per category: AI assistants monthly, browsers quarterly, others six-monthly to annually.
- Public catalogue changelog and data export.
- Documented appeals and correction process.
Catalogue coverage
Services and flags currently in the catalogue. Coverage grows with each research cycle.
| Category | Services | With alternatives | Published flags |
|---|---|---|---|
| 5 | 3 | 85 | |
| Document storage and collaboration | 5 | 3 | 55 |
| Messaging and team chat | 4 | 2 | 33 |
| Video conferencing | 4 | 3 | 34 |
| Content management systems | 6 | 4 | 28 |
| Browsers | 5 | 3 | 30 |
| Search engines | 5 | 4 | 22 |
| Calendar | 4 | 1 | 10 |
| AI assistants | 0 | 0 | 0 |