This week we met with our client Naya to update her on our current progress. We went over the python script that was used to generate the final dataset. This process can be mostly repeated for future datasets but requires entering the name of the dataset files. Many of the data are not clean and require manual attention, this part cannot be directly done through the python script. However, I have included parts where dirty data are exported to a file for cleaning and parts where a cleaned version of the data is imported and merged with the rest of the data.
We will be looking forward to hearing from Naya to see if she has any suggestions for improvement. Currently, we are also developing testing for the accuracy of our project. We plan to inspect whether the location variables correspond with the latitude and longitude; this could suggest that the datasets require further cleaning. We also plan to research methods to speed up the Tableau because currently, the dashboard reacts slowly to some operations.