njr0

20 Flips | 1 Magazine | 2 Likes | 2 Followers | @njr0 | Keep up with njr0 on Flipboard, a place to see the stories, photos, and updates that matter to you. Flipboard creates a personalized magazine full of everything, from world news to life’s great moments. Download Flipboard for free and search for “njr0”

The TDDA Constraints File Format

Background<p>We recently extended the tdda library to include support for automatic discovery of constraints from datasets, and for verification of …

Constraint Discovery and Verification for Pandas DataFrames

Background<p>In a previous post, Constraints and Assertions, we introduced the idea of using constraints to verify input, output and intermediate …

WritableTestCase: Example Use

In my PyCon UK talk yesterday I promised to update the and document the copy of writabletestcase.WritableTestCase on GitHub.<p>The version I've put up …

Slides and Rough Transcript of TDDA talk from PyCon UK 2016

Python UK 2016, Cardiff.<p>I gave a talk on test-driven data analysis at PyCon UK 2016, Cardiff, today.<p>The slides (which are kind-of useless without the …

Extracting More Apple Health Data

The first version of the Python code for extracting data from the XML export from the Apple Health on iOS neglected to extract Activity Summaries and …

Unit Tests

I've added a coding line at the top of both the test script and the main applehealthdata.py script. This tells Python (and my editor, Emacs) the …

First Test

In the last post, I presented some code for extracting (some of) the data from the XML file exported by the Apple Health app on iOS, but—almost …

In Defence of XML: Exporting and Analysing Apple Health Data

I'm going to present a series of posts based around the sort of health and fitness data that can now be collected by some phones and dedicated …

Lessons Learned: Bad Data and other SNAFUs

My first paid programming job was working for my local education authority during the summer. The Advisory Unit for Computer-Based Education (AUCBE), …

How far in advance are flights cheapest? An error of interpretation

<b>Guest Post</b> by Patrick Surry, Chief Data Scientist, Hopper<p>Every year, Expedia and ARC collaborate to publish some annual statistics about domestic …

Generalized Overfitting: Errors of Applicability

Everyone building predictive models or performing statistical fitting knows about <i>overfitting.</i> This arises when the function represented by the model …

Overview of TDDA in Predictive Analytics Times

Constraints and Assertions

Consistency Checking of Inputs, Outputs and Intermediates<p>While the idea of regression testing comes straight from test-driven development, the next …

Infinite Gain: The First Test

Stochastic (Randomized) Analyses<p>We assume that our analytical process is deterministic. If it involves a random component, we can make it …

How is this Misleading Data Misleading Me?

<i>"Why is this lying bastard lying to me?"</i><p>Louis Heren,1 often attributed to Jeremy Paxman.<p>In a previous post, we made a distinction between two kinds …

Test-Driven Development: A Review

Since a key motivation for developing test-driven data analysis (TDDA) has been test-driven development (TDD), we need to conduct a lightning tour of …

Why Test-Driven Data Analysis?

How confident are you that the outputs produced by the analytical process, with the input data you have used, are correct?<p>How confident are you that …

Test-Driven Data Analysis

A dozen or so years ago I stumbled across the idea of <i>test-driven development</i> from reading various posts by Tim Bray on his Ongoing blog. It was …