Submit a two-page report (minimum) describing the process used for preparing your data. Describe data
preparation related to format, normalization, unitization, quality, and cleanliness. Describe the process used for
exploring and modeling your data. Describe data exploration and modeling related to the techniques and
algorithms used. Information modeling is essentially playing with and interpreting the data. Use available tools
such as Excel or Tableau to try to find correlations and relationships. Use visualization techniques to see which
methods reveal interesting information. In many instances, exploratory analysis will show that additional data is
needed, or the data needs additional cleaning or normalization. Consider the following:
1. Prepare/Process Data
• Data initially obtained must be processed or organized for analysis. Data may require some
initial analysis or structuring and relating various data elements.
• Data normalization, e.g., structure data by date ranges, bring monetary figures into
current or future values.
• Map Reduce, Structuring in Tables/Spreadsheets, Natural Language Processing.
2. Clean Data
• Once processed and organized, data and information may be incomplete, contain duplicates, or
contain errors. These errors should be corrected if possible.
• Types and methods of data cleaning will depend on the type of data, such as phone numbers,
email addresses, employers, etc.• Quantitative data methods for outlier detection can be used to get rid of potentially incorrectly
entered data.
• Textual data spell checkers can be used to lessen the amount of mistyped words.
3. Conduct Exploratory Analysis
• Apply a variety of techniques referred to as exploratory data analysis to begin understanding the
relationships, correlations, and messages contained in the data.
• The process of exploration may result in additional data cleaning or additional needs for data, so
these activities may be iterative.
• Algorithms or calculations may be employed, such as the average or median, to help understand
the data. Data visualization may also be used to examine the data in graphical format, to obtain
additional insight into the data and information.
4. Incorporate Modeling & Algorithms
• Post Exploratory Analysis will aid in defining specific algorithms to be implemented in the
analysis.
• Mathematical formulas or models (algorithms) can be applied to the data to identify relationships
among the variables, such as correlation or causation.
• e.g., Clustering algorithm on numerical or textual feature data, Topic Modeling algorithm
on textual data
• Data and information may require additional “normalization.” For instance, interest and
equivalence formulas may be needed to normalize data to a reference year.
• In general terms, models may be developed to evaluate a particular variable in the data based
on other variable(s) in the data, with the possibility of some residual error.
Last Completed Projects
| topic title | academic level | Writer | delivered |
|---|