I’ve been working with water data collected by other people for the past 3 years or so. Typically, I’ve gotten my hands on it in 3 different ways: email, dropbox, etc. from collaborators. Usually in .xlsx or .csv format. Inconsistently formatted; seldom tidy. Downloading manually from a website, e.g. USGS instantaneous flows pre-2007 Using an API for web services such as the water quality portal (WQP) or NWIS. I’ve mostly done this using R functions in the dataRetrieval package put out by USGS.
Continue reading

Breaking the Silence

Haven’t updated this blog in a while. Time to start doing that again. Also, I changed the theme. The last one wasn’t doing it for me. I like this one. It’s called “mnml”
Continue reading


I’m on a train back from Boston after attending the 2015 Open Data Science Conference. Two days of serious nerding out among afficionados of open-source software like R and Python. Herein I give some highlights. Notes from talks and workshops I attended are here Train from boston Some main points: Data scientists do a lot of different things, and it’s still not clear what the job title refers to.
Continue reading

Investigations Geting meta with ranks and order stats. But climate change! It’s become part of my daily ritual to check the updated extended forecast in hopes of seeing of bona fide warm spell on the horizon. And each day my hopes are dashed–30s, some 40s, and now finally some low 50s–but no real birds-a-chirpin’ spring weather. As a son of the Lake Superior shoreline I’m somewhat ashamed to admit my impatience with the relatively mild New England winter, but this one just feels relentless.
Continue reading


I’m currently watching Socrata’s introductory webinar. Thinking about how far the open data movement has to go to in the context of water science. I found myself at the website after browsing the links from a TED talk on NYC open data insights. I had only vaguely been aware of Socrata in the past, but it looks to be catching on: I won’t pretend to know more than I do about Socrata, but I think this graphic pretty well sums up their mission:
Continue reading

Reflections on AGU 2014

It is Saturday morning, the first morning in a week that I’m not groggily shuffling off to the Moscone convention center in San Francisco to fill my mind with geosciencey knowledge. The past week was my first time attending the American Geophysical Union fall meeting–after wanting to go since I was an undergrad in geology. This year’s meeting continued the tradtion of exceeding the previous year’s attendance; 24 thousand of us descended upon the city this year.
Continue reading


I had a thought. I’ve been doing a lot of consuming of academic material over the last couple years–since I started my graduate studies. That means reading (a LOT), taking classes, hearing lectures, and learning research “tools” (software, organization, etc). This sort of thing is slow, time-consuming work that yields a trickle of intellectual returns–i.e. competence. But all in all I like it. I like that apart from convincing myself and (to a much lesser extent) others that I’m digesting and understanding the things I consume, there is very little expectation or evaluation of my abilities.
Continue reading

Author's picture

Mark Hagemann

I think and sometimes write about water or data or other things I think about.