Re-imagining government data in the Philippines
The potential of the National Government Portal (gov.ph)
Before working for Uber, I worked as a data analyst for the Philippines’ Department of Education from 2015 to 2016. One of the reasons I moved was because I was interested in how tech companies operated, hoping to bring back some of those lessons to help the Philippines. This is one attempt at that, specifically dealing with data management and access. The intended audience of this article are policy-makers in the government, specifically those in the Department of Information and Communications Technology, and groups / individuals interested in government data.
My experience in the Philippines
The Department of Education at that time had two major information systems: the Learner Information System (LIS) and the Enhanced Basic Education Information System (EBEIS) (reference). I primarily used the latter for my work to get school-level data. My work-flow consisted of going to the EBEIS user-interface (UI), selecting data filters, downloading the data as comma-separated values (CSV) files, reading the data into Python or Microsoft Excel for processing and analyses, and sharing the results in a deck (I shared initially as Python notebooks but these are not user-friendly). There were a number of problems with these, namely:
- Exclusively CSV-based: All the data files I worked with were in CSVs. Big files could not be handled by Excel, while ingesting these into Python required extra steps not readily available to everyone. Joining data from various files was not as straightforward as it could be.
- Slow to generate files: After submitting the requests on the EBEIS UI, it took a while for the CSV files to be generated (it’s been a while since I used it but, if I remember correctly, it could take more than an hour to generate CSV files with more data). I believe there was a separate team that had access to the underlying EBEIS database in some other way (i.e., not via the UI) and I would reach out to them if I got stuck.
- All data were gated: Individuals that were not employees of the Department of Education did not have access to the EBEIS data (which required a government email). This is perfectly fine for cases dealing with Personal Identifiable Information (PII) data, but it makes it difficult to get high-level data, say, on enrollment numbers of schools.
There’s room for improvement here beyond the Department of Education. For example, in this article by Kenneth Isaiah Ibasco Abante, he got data from the Philippines’ Bureau of Customs, which took him “more than one hundred hours” to clean and document. In order for government agencies and citizens to make data-driven decisions, data has to be more accessible.
The National Government Portal
I encountered the National Government Portal (i.e., gov.ph) while reading the Philippines’ E-government Masterplan 2022 which was developed by the Department of Information and Communications Technology in 2019. The portal serves as “the face of e-government online… It shall be a one-stop shop of government data, information, and services from all government agencies” (p.60, E-government Masterplan 2022). This is an amazing initiative even when just talking about the first part (one-stop shop of government data), let alone the information and service functionalities!
A. Services
Browsing through the portal, the services stand out: these are shown in the portal home page by category (e.g., Business and Trade, Certificates and IDs). Clicking through a category leads to a list of services further divided by sub-categories, and clicking through the items in the list leads to an external website. For example, the category’s (“Business and Trade”) first item (“Apply for a Trademark online”) leads to this external page: http://121.58.254.39:8080/sp-ui-tmefiling/wizard.htm?execution=e1s1 (definitely want this to be a secure site using a URL instead of an IP address).
B. Information
Government information in the portal includes the following:
- a list of government projects: this similar to the services pages where projects are in lists and the items lead to external pages) filterable by agency
- a directory of departments and agencies: this includes the department / agency names, addresses, phone numbers, and email addresses if available (should the availability of an email address for each department / agency. be a metric to monitor?). This is filterable by government branch and by region / city / municipality
- other info: i.e., an overview of the Philippines, hotlines, public holidays, and interoperability standards (referring to the framework the government uses to facilitate e-Government goals). In the home page, there are also links to information / news agencies.
C. Data
I tried looking for available data (e.g., enrollment data) but could not find these in the tabs (still work-in-progress?). I did see the links to Open data, Freedom of Information, and the Philippine business databank at the bottom. Unfortunately, only the Freedom of Information site was up. The Open Data website shows an error message (“Websites sometimes break. Please wait while we put it back together”), while the Philippine business databank was pointing back to the gov.ph homepage (when I checked previously, it was pointing to https://pbdb-staging.apps.gov.ph/#!/app/dashboard/v1 which was inaccessible. Looks like this is also work-in-progress based on the use of a staging URL). I was expecting Open Data not to be available based on a statement in the E-government Masterplan 2022: “data.gov.ph (Open Data) have also migrated to the NGP” (p.44).
My wish list
Considering that the National Government Portal has three functions (i.e., data, information, and services), my wish list only pertains to the data piece, specifically:
Wish #1: Lay out data that are available / will be available
This can be placed in one tab (like the ‘services’ tab). Annex C of the E-government Masterplan 2022 mentions that a target for 2021 is to have “complete OGP [Open Government Partnership] datasets”; it would be helpful to see what these constitutes and which ones are already available.
Wish #2: Include easily readable metadata
As opposed to laying out the available data in lists, this can be done as tables where columns include metadata information: this would extend the utility of the first item in the wish list to inform users what those datasets actually include. To illustrate the utility of this approach, let’s take an example of existing datasets available in the website https://openstat.psa.gov.ph/. Checking databases -> Population and Migration -> Birth Statistics, we see this page:
Laying this out in table format (including metadata not currently shown), it would look like this:
This is easier to read! And it’s easier to know what’s available for a given dataset. If there are questions or issues with the data, additional metadata such as data source and data owner will inform users of appropriate channels to raise concerns. For datasets that are not yet available, the “years available” column will indicate “no data available.”
Wish #3: Provide queryable data
For advanced users, having an option to query the underlying datasets will be useful (e.g., to set up citizen-maintained dashboards). Retrieving the data using SQL is an option that does not rely on UI filters which would provide more flexibility for data retrieval and analyses. Further, this will extend the utility of the second wish list item by including additional metadata columns that can provide insight on data quality (e.g., min / max / avg values of fields).
Unlike the other items in the wishlist, this requires a review of how the data are stored by departments and agencies, as well as how the data can be accessed by users (e.g., how should permissions be managed to ensure PII data are not revealed). Open source solutions for this are available such as those used by Uber: https://eng.uber.com/cost-efficient-big-data-platform/.
Wish #4: Provide an easy way for users to give feedback
“Users” here refers to anyone using the portal and are interested in looking at data. The feedback here could pertain to data that should be included, data that should be updated, and questions / concerns on the data. Two ways to achieve this:
- discussion forum: The portal could included a discussion forum for data concerns (potentially concerns relating to information and services as well). This has to be moderated and the feedback collated before it can be used, however. A cleaner way to get feedback is described below.
- system for filing data support tickets: The portal currently has a way to provide general feedback, but having a specific system for filing support tickets for data concerns would help streamline this process. In addition, the filed tickets should be surfaced in a page in the portal so users can see what concerns have been raised already; having a way to vote on tickets that require more attention would be even more helpful.
You can reach me at LinkedIn: https://www.linkedin.com/in/likas-umali-91a469183
