Our methodology
This architecture is meant as a POC for the AI for Impact hackathon 2024 by Google. We hope to one day refine and scale this solution for all government datasets around the world.
Data Sourcing & Extraction
We work with publicly available data sources such as data.gov.sg to extract datasets which are usable for generating our reports. These datasets are often used by the public, journalists, international or government agencies for the purposes of tracking progress and providing transparency.
They have to be verified and secure sources in order for us to ensure the accuracy of the information we present and feed into the AI tools.
Most of these agencies have documentation on APIs to extract information that we first store in order to feed into various AI models. The bridge we want to build is then how might we make this information more readable.
Leveraging AI for Scale
We view AI primarily as a compositional tool - leveraging on it's ability to restructure information in a more intuitive and utilizable format.
With some prompt engineering, we built prompts to create:
1. Visualizations depending on the nature/format of the dataset.
2. A summary insight that describes trends, anomalies or patterns in the information of note
3. Methodology on how the information is put together in the Visualization and how the insights are derived.
Improving discoverability through AI-based Search
Utilizing Vertex AI from Google Search to put search queries into context and point to relevant reports or insights.
Collaborative Development
By reviewing feedback on the reports we generate - we want to make these report generation more robust. Additionally, we are hoping that these reports are just a starting position for more users to develop their own insights reports that they can share with us and the wider world.
Scalable Architecture
We wanted to make this platform plug-and-play for multiple governments and data sources. Thus we leveraged on Google Cloud's suite of services in order to deliver a scalable and adaptable platform.
Principles for report generation and insights submission
1. Verifiability
Each report or visualization will be replicated in order to determine that the data presented is aligned with the conclusions drawn.
2. Good Faith
Any report which is framed in such a way that may result in discourse or harm to public interest may be rejected.
3. Clarity
Masking explanations of conclusions drawn behind complex methodology goes against this principle. However, we celebrate technical expertise which bring simplicity to complex problems.
4. Objectivity
Even while acting in good faith, we remain dedicated to being objective in the information we present. Providing clarity on what is certain and what is not is vital to ensure we maintain objectivity in these reports.
5. Accountability
While we keep your information private, all authors should be accountable to their work. This extends to ensuring that the work you produce is not plagiarized and properly attributed where necessary.
All information published on this site adheres to our core principles of verifiability, good faith, clarity, objectivity and accountability. By upholding these standards, we ensure that each contribution aligns with a common commitment to transparency and public interest. This approach allows diverse insights to coexist, creating a trustworthy platform where citizens can access and understand data-driven perspectives on our nation’s development
You can contribute too
.svg){kind=icon}