There’s been a fair bit of discussion over on Micro.blog about podcast players recently. I’ve switched among Overcast, Castro, and Apple Podcasts players over the years and, mostly to help myself think it through (again), here are my thoughts.
For me, there are three main criteria: audio quality, episode management, and OS integration. Though, I completely understand that others may have different criteria.
Like most podcast listeners, I listen at high speed, usually around 1.5x, which can distort the audio. Apple Podcasts player is definitely the worst for this criterion. For me, Overcast’s smart speed and voice boost features give it the edge over Castro, in terms of audio quality at increased speeds.
Castro is definitely the player of choice if you subscribe to more podcasts than you can listen to. The queue management features in Castro are very good. You can replicate some of this with Overcast, but that isn’t one of the app’s main features. Apple Podcasts player doesn’t offer much in this regard. Despite listing this as a criterion, I’ve actually elevated to subscription, rather than episode, triage. I carefully curate my list of podcasts and don’t add new ones very often. Given this, Overcast’s approach is more than sufficient for me.
OS integration is a bit unfair, as Apple has given themselves some nice features that independent app developers aren’t able to use. Nonetheless, Apple does a good job of intermixing podcasts and music into their Siri-powered widgets in iOS and the HomePods. With my reinforced interest in less screen time and use of home screen widgets, Apple Podcasts wins out here. This may change once Overcast releases a widget.
As just one example of the integration, here’s what happens when I plug headphones into my iPhone: a selection of music playlists and podcasts appears. I use this feature a lot.
After all that, I’m using Apple Podcasts for now, mostly to take advantage of the OS integrations while I’m experimenting with a widget-based iPhone. I’m quite certain this is short term and that I’ll return to Overcast soon. In addition to the better audio quality, Overcast also has some nice refinements, like per subscription speed settings (I like to play music podcasts at 1x) and skip-forward amounts (Quirks and Quarks, for example, always has a two-minute preamble that I skip). These small refinements are typically what distinguishes the stock Apple apps from good indie apps.
I’m glad there are so many solid apps for podcast listening. Whatever your preferences, there’s sure to be one for you.
Like any 12-year old, my son is pretty keen on gaming. As an all Apple house, his options were a bit constrained. So, we decided to build a PC from components.
I’d last built a PC about 30 years ago, when I wasn’t much older than him. I remember thinking it was cool to be using a machine I’d built myself, plus as a parent it seemed like a good educational experience. I have to admit to being a bit nervous about the whole thing, as there was certainly a scenario in which we spent an entire weekend unsuccessfully trying to get a bunch of malfunctioning components to work.
After much deliberation and analysis, we ordered our parts from Newegg and everything arrived within a couple of weeks.
Given my initial anxiety, I was very relieved when we saw this screen. The BIOS booted up and showed that the RAM, SSD, and other components were all properly connected.
With that done, we then got to what ended up being the complicated part. Evidently part of the point of a gaming PC is to have lots of fans and lights. None of this was true when I was a kid and there were a daunting number of wires required to power the lights and fans. Sorting this out actually took a fair bit of time. But, eventually success!
Then our last challenge, which I likely should have anticipated much sooner. We didn’t bother ordering a DVD drive, since everything is online these days. But, our Windows installation showed up as a DVD and we couldn’t create any Windows install media on our Apple devices. Fortunately, we checked in with a slightly older kid down the street and he provided us with a USB drive with the right software. With that challenge solved, we finished the project!
Not counting choosing the components online, the whole project took about 5 hours from opening the boxes to booting into Windows for the first time. I’d definitely recommend it to anyone that’s tempted and technically inclined. My son is quite excited to be using a computer that he built from parts.
Staying in touch with my team is important. So, I schedule a skip-level meeting with someone on the team each week. These informal conversations are great for getting to know everyone, finding out about new ideas, and learning about recent achievements.
Getting these organized across a couple of dozen people is logistically challenging and I’ve developed a Shortcut to automate most of the process.
Borrowing from Scotty Jackson, I have a base in AirTable with a record for each team member. I use this to store all sorts of useful information about everyone, including when we last had a skip-level meeting. The Shortcut uses this field to pull out team members that I haven’t met with in the past four months and then randomizes the list of names. Then it passes each name over to Fantastical while also incrementing the date by a week. The end result is a recurring set of weekly meetings, randomized across team members.
The hardest part of the Shortcut development was figuring out how to get the names in a random order. A big thank you to sylumer in the Automators forum for pointing out that the Files action can randomly sort any list, not just lists of files.
I’m not sharing the Shortcut here, since the implementation is very specific to my needs. Rather, I’m sharing some of the thinking behind the code, since I think that it demonstrates the general utility of something like Shortcuts for managing routine tasks with just a small amount of upfront effort.
Inspired by Coretex, I’m declaring Tangible as my theme for 2021.
I’ve chosen this theme because I want to spend less time looking at a screen and more time with “tangible stuff”. I’m sure that this is a common sentiment and declaring this theme will keep me focused on improvements.
Since working from home with an iPad, I’m averaging about 9 hours a day with an iOS device. This isn’t just a vague estimate; Screen Time gives me to-the-minute tracking of every app I’m actively using.
I’m certainly not a Luddite! The ability of these rectangles of glass to take on so many functions and provide so much meaningful content is astounding. There’s just something unsettling about the dominant role they play.
So, a few things I plan to try:
Although I’ll continue reading ebooks, since the convenience is so great, I’ll be rotating paper books into the queue regularly.
I’ve lost my running outdoor routine. Getting that back will be a nice addition to the Zoom classes and add in some very much needed fresh air.
As a family, we’ve been enjoying playing board games on weekends, just not routinely. Making sure we play at least one game a week will be good for all of us.
My son is keen on electronics. We’re going to try assembling a gaming PC for him from components, as well as learn some basic electronics with a breadboard and Arduino.
Our dog will be excited to get out for more regular walks with especially long ones on the weekend.
I’ll be adding much of this to Streaks, an app that I’ve found really helpful for building habits. I’ll also add a “tangible” tag to my time tracker to quantify the shift.
My hope is that I can find the right balance of screen time and tangible activities with intention.
MindNode is indispensable to my workflow. My main use for it is in tracking all of my projects and tasks, supported by MindNode’s Reminders integration. I can see all of my projects, grouped by areas of focus, simultaneously which is great for weekly reviews and for prioritizing my work.
I’ve also found it really helpful for sketching out project plans. I can get ideas out of my head easily with quick entry and then drag and drop nodes to explore connections. Seeing connections among items and rearranging them really brings out the critical elements.
MindNode’s design is fantastic and the app makes it really easy to apply styles across nodes. The relatively recent addition of tags has been great too. Overall, one of my most used apps.
I’m very excited to be recruiting for a Data Governance Sponsor to join my team and help enhance the use of good data analytics in our decisions at Metrolinx.
I’m looking for someone that enjoys telling compelling stories with data and has a passion for collaborating to build clean and reliable analytical processes. If you know someone that could fit (maybe you!), please pass along the job ad
I’ve been negligent in supporting some of my favourite apps on the App Store. In many cases, I reviewed the app a few years ago and then never refreshed my ratings. So, I’m making a new commitment to updating my reviews for apps by picking at least one each month to refresh.
First up is Fantastical. This one took a real hit when they switched to a subscription pricing model. I get the controversy with subscriptions in general. For me, Fantastical has earned a spot on my short list of apps that I support with an ongoing subscription.
And here’s my App Store review:
Fantastical is a great app and is definitely one of my top three most-used apps. Well worth the subscription price.
A few favourite features:
Integration of events and tasks into the calendar view
Access to event attachments
Automatic link detection for Teams and Zoom meetings
With the release of iOS 7, I’m reconsidering my earlier approach to the Home Screen. So far I’m trying out a fully automated first screen that uses the Smart Stack, Siri Suggestions, and Shortcut widgets. These are all automatically populated, based on anticipated use and have been quite prescient.
My second screen is all widgets with views from apps that I want to have always available. Although the dynamic content on the first screen has been really good, I do want some certainty about accessing specific content. This second screen replaces how I was using the Today View. I’m not really sure what to do with that feature anymore.
I’ve hidden all of the other screens and rely on the App Library and search to find anything else.
I still like the simplicity behind my earlier approach to the Home Screen. We’ll see if that is just what I’m used to. This new approach is worth testing out for at least a few weeks.
Skipping past the unnecessarily dramatic title, The Broken Algorithm That Poisoned American Transportation does make some useful points. As seems typical though these days, the good points are likely not the ones a quick reader would take away. My guess is most people see the headline and think that transportation demand models (TDMs) are inherently broken. Despite my biases, I don’t think this is actually true.
For me, the most important point is about a third of the way through:
nearly everyone agreed the biggest question is not whether the models can yield better results, but why we rely on them so much in the first place. At the heart of the matter is not a debate about TDMs or modeling in general, but the process for how we decide what our cities should look like.
Models are just a tool for helping guide decisions. Ideally we would use them to compare alternatives and pick a favoured “vector” of change (rough direction and magnitude). Then with continuous monitoring and refinements throughout the project’s lifecycle, we can guide decisions towards favoured outcomes. This is why scenario planning, sensitivity tests, and clear presentation of uncertainty are so important. This point is emphasized later in the article:
civil engineers doing the modeling tend to downplay the relevance of the precise numbers and speak more broadly about trends over time. Ideally, they argue, policymakers would run the model with varying population forecasts, land use patterns, and employment scenarios to get a range of expectations. Then, they would consider what range of those expectations the project actually works for.
Although I’m not a civil engineer, this sounds right to me! I get that people want certainty and precise numbers, I just don’t think anyone can provide these things. Major infrastructure projects have inherent risks and uncertainty. We need to acknowledge this and use judgement, along with a willingness to adjust over time. There is no magical crystal ball that can substitute for deliberation. [Me working from home:🧙♂️🔮]
Fortunately for the modellers among us, the article does acknowledge that we’re getting better:
As problematic as they have been, the models have gotten smarter. Especially in the last decade or so, more states are working from dynamic travel models that more closely reflect how humans actually behave. They are better at taking into consideration alternate modes of transportation like biking, walking, and public transportation. And, unlike previous versions, they’re able to model how widening one section of road might create bottlenecks in a different section.
But, wait:
Still, experts warn that unless we change the entire decision-making process behind these projects, a better model won’t accomplish anything. The models are typically not even run—and the results presented to the public—until after a state department of transportation has all but settled on a preferred project.
😔 Maybe it wasn’t the model’s fault after all.
This brings as back to the earlier point: we should be favouring more sophisticated decision-making processes, not just more sophisticated models.
I haven’t yet adopted the minimalist style of my iPhone for my iPad. Rather, I’ve found that setting up “task oriented” Shortcuts on my home screen is a good alternative to arranging lots of app icons.
The one I use the most is a “Reading” Shortcut, since this is my dominant use of the iPad. Nothing particularly fancy. Just a list of potential reading sources and each one starts up a Timery timer, since I like to track how much time I’m reading.
A nice feature of using a Shortcut for this is that I can add other actions, such as turning on Do Not Disturb or starting a specific playlist. I can also add and subtract reading sources over time, depending on my current habits. For example, the first one was Libby for a while, since I was reading lots of library books.
This is another example of how relatively simple Shortcuts can really help optimize how you use your iOS devices.
I’ve been keeping a “director’s commentary” of my experiences in Day One since August 2, 2012 (5,882 entries and counting). I’ve found this incredibly helpful and really enjoy the “On This Day” feature that shows all of my past entries on a particular day.
For the past few months, I’ve added in a routine based on the “5 minute PM” template which prompts me to add three things that happened that day and one thing I could have done to make the day better. This is a great point of reflection and will build up a nice library of what I’ve been doing over time.
My days seem like such a whirlwind sometimes that I actually have trouble remembering what I did that day. So, my new habit is to scroll through my Today view in Agenda. This shows me all of my notes from the day’s meetings. I’ve also created a Shortcut that creates a new note in Agenda with all of my completed tasks from Reminders. This is a useful reminder of any non-meeting based things I’ve done (not everything is a meeting, yet).
I’m finding this new routine to be a very helpful part of my daily shutdown routine: I often identify the most important thing to do tomorrow by reviewing what I did today. And starting tomorrow off with my top priority already identified really helps get the day going quickly.
watchOS 7 has some interesting new features for enhancing and sharing watch faces. After an initial explosion of developing many special purpose watch faces, I’ve settled on two: one for work and another for home.
Both watch faces use the Modular design with the date on the top left, time on the top right, and Messages on the bottom right. I like keeping the faces mostly the same for consistency and muscle memory.
My work watch face than adds the Fantastical complication right in the centre, since I often need to know which meeting I’m about to be late for. Reminders is on the bottom left and Mail in the bottom centre. I have this face set to white to not cause too much distraction.
My home watch face swaps in Now Playing in the centre, since I’m often listening to music or podcasts. And I have Activity in the bottom centre. This face is in orange, mostly to distinguish it from the work watch face.
Surprisingly, I’ve found this distinction between a work and home watch face even more important in quarantine. Switching from one face to another really helps enforce the transition between work and non-work when everything is all done at home.
The watch face that I’d really like to use is the Siri watch face. This one is supposed to intelligently expose information based on my habits. Sounds great, but almost never actually works.
Our predictions for the 2019 Federal race in Toronto were generated by our agent-based model that uses demographic characteristics and results from previous elections. Now that the final results are available, we can see how our predictions performed at the Electoral District level.
For this analysis, we restrict the comparison to just the major parties, as they were the only parties for which we estimated vote share. We also only compare the actual results to the predictions of our base scenario. In the future, our work will focus much more on scenario planning to explain political campaigns.
We start by plotting the difference between the actual votes and the predicted votes at the party and district level.
Distribution of the difference between the predicted and actual proportion of votes for all parties
The mean absolute value of differences from the actual results is 5.3%. In addition, the median value of the differences is 1.28%, which means that we slightly overestimated support for parties. However, as the histogram shows, there is significant variation in this difference across districts. Our highest overestimation was 15.6% and lowest underestimation was -18.5%.
To better understand this variation, we can look at a plot of the geographical distribution of the differences. In this figure, we show each party separately to illuminate the geographical structure of the differences.
Geographical distribution of the difference between the predicted and actual proportion of votes by Electoral District and party
The overall distribution of differences doesn’t have a clear geographical bias. In some sense, this is good, as it shows our agent-based model isn’t systematically biased to any particular Electoral District.
However, our model does appear to generally overestimate NDP support while underestimating Liberal support. These slight biases are important indicators for us in recalibrating the model.
Overall, we’re very happy with an error distribution of around 5%. As described earlier, our primary objective is to explain political campaigns. Having accurate predictions is useful to this objective, but isn’t the primary concern. Rather, we’re much more interested in using the model that we’ve built for exploring different scenarios and helping to design political campaigns.
I’m neither an epidemiologist nor a medical doctor. So, no one wants to see my amateur disease modelling.
That said, I’ve complained in the past about Ontario’s open data practices. So, I was very impressed with the usefulness of the data the Province is providing for COVID: a straightforward csv file that is regularly updated from a stable URL.
Using the data is easy. Here’s an example of creating a table of daily counts and cumulative totals:
Since I’m mostly stuck inside these days, I find I’m drinking more tea than usual. So, as a modification of my brew coffee shortcut, I’ve created a brew tea shortcut.
This one is slightly more complicated, since I want to do different things depending on if the tea is caffeinated or not.
We start by making this choice:
Then, if we choose caffeine, we log this to the Health app:
Uncaffeinated tea counts as water (at least for me):
And, then, regardless of the type of tea, we set a timer for 7 minutes:
Running this one requires more interactions with Siri, since she’ll ask which type we want. We can either reply by voice or by pressing the option we want on the screen.
I’ve been tracking my time at work for a while now, with the help of Toggl and Timery. Now that I’m working from home, work and home life are blending together, making it even more useful to track what I’m doing.
Physical exercise is essential to my sanity. So, I wanted to integrate my Apple Watch workouts into my time tracking. I thought I’d be able to leverage integration with the Health app through Shortcuts to add in workout times. Turns out you can’t access this kind of information and I had to take a more indirect route using the Automation features in Shortcuts.
I’ve setup two automations: one for when I start an Apple Watch workout and the other for when I stop the workout:
The starting automation just starts an entry in Timery:
The stopping automation, unsurprisingly, stops the running entry:
As with most of my Shortcuts, this is a simple one. Developing a portfolio of these simple automations is really helpful for optimizing my processes and freeing up time for my priorities.
That said, it is often the smaller automations that add up over time to make a big difference. My most used one is also the simplest in my Shortcuts Library. I use it every morning when I make my coffee. All the shortcut does is set a timer for 60 seconds (my chosen brew time for the Aeropress) and logs 90mg of caffeine into the Health app.
All I need to do is groggily say “Hey Siri, brew coffee” and then patiently wait for a minute. Well, that plus boil the water and grind the beans.
Simple, right? But that’s the point. Even simple tasks can be automated and yield consistencies and productivity gains.
I’m delivering a seminar on estimating capital costs for large transit projects soon. One of the main concepts that seems to confuse people is inflation (including the non-intuitive terms nominal and real costs). To guide this discussion, I’ve pulled data from Statistics Canada on the Consumer Price Index (CPI) to make a few points.
The first point is that, yes, things do cost more than they used to, since prices have consistently increased year over year (this is the whole point of monetary policy). I’m illustrating this with a long-term plot of CPI in Canada from 1914-01-01 to 2019-11-01.
I added in the images of candy bars to acknowledge my grandmother’s observation that, when she was a kid, candy only cost a penny. I also want to make a point that although costs have increased, we also now have a much greater diversity of candy to choose from. There’s an important analogy here for estimating the costs of projects, particulary those with a significant portion of machinery or technology assets.
The next point I want to make is that location matters, which I illustrate with a zoomed in look at CPI for Canada, Ontario, and Toronto.
This shows that over the last five years Toronto has seen higher price increases than the rest of the province and country. This has implications for project costing, since we may need to consider the source of materials and location of the project to choose the most appropriate CPI adjustment.
The last point I want to make is that the type of product also matters. To start, I illustrate this by comparing CPI for apples and alcoholic beverages (why not, there are 330 product types in the data and I have to pick a couple of examples to start).
In addition to showing how relative price inflation between products can change over time (the line for apples crosses the one for alcoholic beverages several times), this chart shows how short-term fluctuations in price can also differ. For example, the line for apples fluctuates dramatically within a year (these are monthly values), while alcoholic beverages is very smooth over time.
Once I’ve made the point with a simple example, I can then follow up with something more relevant to transit planners by showing how the price of transportation, public transportation, and parking have all changed over time, relative to each other and all-items (the standard indicator).
At least half of transit planning seems to actually be about parking, so that parking fees line is particularly relevant.
Making these charts is pretty straightforward, the only real challenge is that the data file is large and unwieldy. The code I used is here.
Podcasts are great. I really enjoy being able to pick and choose interesting conversations from such a broad swath of topics. Somewhere along the way though, I managed to subscribe to way more than I could ever listen to and the unlistened count was inducing anxiety (I know, a real first world problem).
So, time to start all over again and only subscribe to a chosen few:
Quirks & Quarks is the one I’ve been subscribed to the longest and is a reliable overview of interesting science stories. I’ve been listening to this one for so long that I used to rely on an AppleScript to get episodes into my original scroll wheel iPod, well before podcasts were embraced by Apple.
In Our Time is another veteran on my list. I really like the three-academics format and Melvyn Bragg is a great moderator. This show has a fascinating diversity of topics in science, history, and literature.
All Songs Considered has helped me keep up with the latest music and Bob Boilen is a very good interviewer.
The Talk Show has kept me up to date on the latest in Apple and related news since at least 2007.
Exponent has really helped me think more clearly about strategy with discussions of tech and society.
Focused has been a very helpful biweekly reminder to think more carefully about what I’m working on and how to optimize my systems.
Making Sense has had reliably interesting discussions from Sam Harris. It just recently went behind a paywall. But I’m happy to pay for it, which comes with access to the Waking Up app.
I admire what Jesse Brown has built with CANADALAND and happily support it.
Mindscape has had some of the most interesting episodes of any of my subscriptions in the last several months. There’s definitely a bias towards quantum mechanics and physics, but there’s nothing wrong with that.
When all together on a list like this, it looks like a lot. Many are biweekly though, so they don’t accumulate.
I use Overcast for listening to these. I’ve tried many other apps and this one has the right mix of features and simplicity for me. I also appreciate the freedom of the Apple Watch integration which allows me to leave my phone at home and still have podcasts to listen to.
As outlined in our last two posts, our algorithm has “learned” how to simulate the behavioural traits of over 2 million voters in Toronto. This allows us to turn their behavioural “dials” and see what happens.
To demonstrate, we’ll simulate three scenarios:
The “likeability” of the Liberal Party falls by 10% from the baseline (i.e., continues to fall);
The Conservative Party announces a policy stance regarding climate change much more aligned with the other parties; and
People don’t vote strategically and no longer consider the probability of each candidate winning in their riding (i.e., they are free to vote for whomever they align with and like the most, somewhat as if proportional representation were a part of our voting system).
Let’s examine each scenario separately:
1 – If Liberal “likeability” fell
In this scenario, the “likeability” scores for the Liberals in each riding falls by 10% (the amount varies by riding). This could come from a new scandal (or increased salience and impact of previous ones).
What we see in this scenario is a nearly seven point drop in Liberal support across Toronto, about half of which would be picked up by the NDP. This would be particularly felt in certain ridings that are already less aligned on policy where changes in “likeability” have a greater impact. The Libs would only safely hold 13/25 seats, instead of 23/25.
From a seat perspective, the NDP would pick up another seat (for a total of three) in at least 80% of our simulations – namely York South-Weston. (It would also put four – Beaches-East York, Davenport, Spadina-Fort York, and University-Rosedale – into serious play.) Similarly, the Conservatives would pick up two seats in at least 80% of our simulations – namely Eglinton-Lawrence and York Centre (and put Don Valley North, Etobicoke Centre, and Willowdale into serious play).
This is a great example of how changing non-linear systems can produce results that are not linear (meaning they cannot be easily predicted by polls or regressions).
2 – If Conservatives undifferentiated themselves on climate change
In this scenario, the Conservatives announce a change to their policy position on a major issue, specifically climate change. The salience of this change would be immediate (this can also be changed, but for simplicity we won’t do so here). It may seem counterintuitive, but it appears that the Conservatives, by giving up a differentiating factor, would actually lose voters. Specifically, in this scenario, no seats change hands, but the Conservatives actually give up about three points to the Greens.
To work this through, imagine a voter who may like another party more, but chooses to vote Conservative specifically because their positions on climate change align. But if the party moved to align its climate change policy with other parties, that voter may decide that there is no longer a compelling enough reason to vote Conservative. If there are more of these voters than voters the party would pick up by changing this one policy (e.g., because there are enough other policies that still dissuade voters from shifting to the Conservatives), then the Conservatives become worse off.
The intuition may be for the defecting Conservative voters discussed above to go Liberal instead (and some do), but in fact, once policies look more alike, “likeability” can take over, and the Greens do better there than the Liberals.
This is a great example of how the emergent properties of a changing system cannot be seen by other types of models.
Proportional Representations
Recent analysis done by P.J. Fournier (of 338Canada) for Macleans Magazine used 338Canada’s existing poll aggregations to estimate how many seats each party would win across Canada if (at least one form of) proportional representation was in place for the current federal election. It is an interesting thought experiment and allows for a discussion of the value of changing our electoral practice.
As supportive as we are of such analysis, this is an area of analysis perfectly set up for agent-based modeling. That’s because Fournier’s analysis assumes no change in voting behavior (as far as we can tell), whereas ABM can relax that assumption and see how the algorithm evolves.
To do so, we have our voters ignore the winning probabilities of each candidate and simply pick who they would want to (including their “likeability”).
Perhaps surprisingly, the simulations show that the Liberals would lose significant support in Toronto (and likely elsewhere). They would drop to third place, behind the Conservatives (first place) and the Greens (second place).
Toronto would transform into four-party city: depending on the form of proportional representation chosen, the city would have 9-12 Conservative seats, 4-7 Green seats, 2-5 Liberal seats, and 2-3 NDP seats.This suggests that most Liberal voters in Toronto are supportive only to avoid their third or fourth choice from winning. This ties in with the finding that Liberals are not well “liked” (i.e., outside of their policies), and might also suggest why the Liberals back-tracked on electoral reform – though such conjecture is outside our analytical scope. Nonetheless, it does support the idea that the Greens are not taken seriously because voters sense that the Greens are not taken seriously by other voters.
More demonstrations are possible
Overall, these three scenarios showcase how agent-based modeling can be used to see the emergent outcomes of various electoral landscapes. Many more simulations could be run, and we welcome ideas for things that would be interesting to the #cdnpoli community.
In our last post, our analysis assumed that voters had a very good sense of the winning probabilities for each candidate in their ridings. This was probably an unfair assumption to make - voters have a sense of which two parties might be fighting for the seat, but unlikely that they know the z-scores based on good sample size polls.
So, we’ve loosened that statistical knowledge a fair amount, whereby voters only have some sense of who is really in the running in their ridings. While that doesn’t change the importance of “likeability” (still averaging around 50% of each vote), it does change which parties' votes are driven by “likeability” more than their policies.
Now, it is in fact the Liberals who fall to last in “likeability” - and by a fairly large margin - coming last or second last in every riding. This suggests that a lot of people are willing to hold their nose and vote for the Libs.
On average, the other three parties have roughly equal “likeability”, but this is more concentrated for some parties than for others. For example, the Greens appear to be either very well “liked” or not “liked” at all. They are the most “liked” in 13/25 ridings and least “liked” in 9/25 ridings - and have some fairly extreme values for “likeability”. This would suggest that some Green supporters are driven entirely by policy while others are driven by something else.
The NDP and Conservatives are more consistent, but the NDP are most “liked” in 10/25 ridings whereas the Conservatives are most “liked” in the remaining 2/25 ridings.
As mentioned in the last post, we’ll be posting some scenarios soon.
The goal of PsephoAnalytics is to model voting behaviour in order to accurately explain political campaigns. That is, we are not looking to forecast ongoing campaigns – there are plenty of good poll aggregators online that provide such estimation. But if we can quantitatively explain why an ongoing campaign is producing the polls that it is, then we have something unique.
That is why agent-based modeling is so useful to us. Our model – as a proof of concept – can replicate the behaviour of millions of individual voters in Toronto in a parameterized way. Once we match their voting patterns to those suggested by the polls (specifically those from CalculatedPolitics, which provides riding-level estimates), we can compare the various parameters that make up our agents behaviour and say something about them.
We can also, therefore, turn those various behavioural dials and see what happens. For example, what if a party changed its positions on a major policy issue, or if a party leader became more likeable? That allows us to estimate the outcomes of such hypothetical changes without having to invest in conducting a poll.
Investigating the 2019 Federal Election
As in previous elections, we only consider Toronto voters, and specifically (this time) how they are behaving with respect to the 2019 federal election. We have matched the likely voting outcomes of over 2 million individual voters with riding-level estimates of support for four parties: Liberals, Conservatives, NDP, and Greens. This also means that we can estimate the response of voters to individual candidates, not just the parties themselves.
First, let’s start with the basics – here are the likely voter outcomes by ridings for each party, as estimated by CalculatedPolitics on October 16.
As these maps show, the Liberals are expected to win 23 of Toronto’s 25 ridings. The two exceptions are Parkdale-High Park and Toronto-Danforth, which are leaning NDP. Four ridings, namely Eglinton-Lawrence, Etobicoke Centre, Willowdale, and York Centre, see the Liberals slightly edging out the Conservatives. Another four ridings, namely Beaches-East York, Davenport, University-Rosedale, and York South-Weston, see the Liberals slightly edging out the NDP. The Greens do no better than 15% (Toronto Danforth), average about 9% across the city, and are highly correlated with support for the NDP.
What is driving these results? First, a reminder about some of the parameters we employ in our model. All “agents” (e.g., voters, candidates) take policy positions. For voters, these are estimated using numerous historical elections to derive “natural” positions. For candidates, we assign values based on campaign commitments (e.g., from CBC’s coverage, though we could also simply use a VoteCompass). Some voters can also care about policy more than others, meaning they care less about non-policy factors (we use the term “likeability” to capture all these non-policy factors). As such, candidates also have a “likeability” score. Voters also have an “engagement” score that indicates how likely they are to pay attention to the campaign and, more importantly, vote at all. Finally, voters can see polls and determine how likely it is that certain parties will win in their riding. Each voter then determine, for each party a) how closely is their platform aligned with the voter’s issue preferences; b) how much do they “like” the candidate (for non-policy reasons); and c) how likely is it the candidate can win in their riding. That information is used by the voter to score each candidate, and then vote for the candidate with the highest score, if the voter chooses to vote at all. (There are other parameters used, but these few provide much of the differentiation we see.)
Based on this, there are a couple of key take-aways from the 2019 federal election:
“Likeability” is important, with about 50% of each vote, on average, being determined by how much the voter likes the party. The importance of “likeability” ranges from voter to voter (extremes of 11% and 89%), but half of voters use “likeability” to determine somewhere between 42% and 58% of their vote.
Given that, some candidates are simply not likeable enough to overcome a) their party platforms; or b) their perceived unlikelihood of victory (over which they have almost no control). For example, the NDP have the highest average “likeability” scores, and rank first in 18 out of 25 ridings. By contrast, the Greens has the lowest average. This means that policy issues (e.g., climate change) are disproportionately driving Green Party support, whereas something else (e.g., Jagmeet Singh’s popularity) is driving NDP support.
In our next post, we’ll look at some scenarios where we change some of these parameters (or perhaps more drastic things).
For several years now, I’ve been a very happy Things user for all of my task management. However, recent reflections on the nature of my work have led to some changes. My role now mostly entails tracking a portfolio of projects and making sure that my team has the right resources and clarity of purpose required to deliver them. This means that I’m much less involved in daily project management and have a much shorter task list than in the past. Plus, the vast majority of my time in the office is spent in meetings to coordinate with other teams and identify new projects.
As a result, in order to optimize my systems, I’ve switched to using a combination of MindNode and Agenda for my task managment.
MindNode is an excellent app for mind mapping. I’ve created a mind map that contains all of my work-related projects across my areas of focus. I find this perspective on my projects really helpful when conducting a weekly review, especially since it gives me a quick sense of how well my projects are balanced across areas. As an example, the screenshot below of my mind map makes it very clear that I’m currently very active with Process Improvement, while not at all engaged in Assurance. I know that this is okay for now, but certainly want to keep an eye on this imbalance over time. I also find the visual presentation really helpful for seeing connections across projects.
MindNode has many great features that make creating and maintaining mind maps really easy. They look good too, which helps when you spend lots of time looking at them.
Agenda is a time-based note taking app. MacStories has done a thorough series of reviews, so I won’t describe the app in any detail here. There is a bit of a learning curve to get used to the idea of a time-based note, though it fits in really well to my meeting-dominated days and I’ve really enjoyed using it.
One point to make about both apps is that they are integrated with the new iOS Reminders system. The new Reminders is dramatically better than the old one and I’ve found it really powerful to have other apps leverage Reminders as a shared task database. I’ve also found it to be more than sufficient for the residual tasks that I need to track that aren’t in MindNode or Agenda.
I implemented this new approach a month ago and have stuck with it. This is at least three weeks longer than any previous attempt to move away from Things. So, the experiment has been a success. If my circumstances change, I’ll happily return to Things. For now, this new approach has worked out very well.
RStudio Cloud is a great service that provides a feature-complete version of RStudio in a web browser. In previous versions of Safari on iPad, RStudio Cloud was close to unusable, since the keyboard shortcuts didn’t work and they’re essential for using RStudio. In iPadOS, all of the shortcuts work as expected and RStudio Cloud is completely functional.
Although most of my analytical work will still be on my desktop, having RStudio on my iPad adds a very convenient option. RStudio Cloud also allows you to setup a project with an environment that persists across any device. So, now I can do most of my work at home, then fix a few issues at work, and refine at a coffee shop. Three different devices all using the exact same RStudio project.
One complexity with an RStudio Cloud setup is GitHub access. The usual approach of putting your git credentials in an .REnviron file (or equivalent) is a bad idea on a web service like RStudio Cloud. So, you need to type your git credentials into the console. To avoid having to do this very frequently, follow this advice and type this into the console:
My goal for the home screen is to stay focused on action by making it easy to quickly capture my intentions and to minimize distractions. With previous setups I often found that I’d unlock the phone, be confronted by a screen full of apps with notification badges, and promptly forget what I had intended to do. So, I’ve reduced my home screen to just two apps.
Drafts is on the right and is likely my most frequently used app. As the tag line for the app says, this is where text starts. Rather than searching for a specific app, launching it, and then typing, Drafts always opens up to a blank text field. Then I type whatever is on my mind and send it from Drafts to the appropriate app. So, text messages, emails, todos, meeting notes, and random ideas all start in Drafts. Unfortunately my corporate iPhone blocks iCloud Drive, so I can’t use Drafts to share notes across my devices. Anything that I want to keep gets moved into Apple Notes.
Things is on the left and is currently my favoured todo app. All of my tasks, projects, and areas of focus are in there, tagged by context, and given due dates, if appropriate. If the Things app has a notification badge, then I’ve got work to do today. If you’re keen, The Sweet Setup has a great course on Things.
A few more notes on my setup:
If Drafts isn’t the right place to start, I just pull down from the home screen to activate search and find the right app. I’ve found that the Siri Suggestions are often what I’m looking for (based on time of day and other context).
Some apps are more important for their output than input. These include calendar, weather, and notes. I’ve set these up as widgets in the Today View. A quick slide to the right reveals these.
I interact with several other apps through notifications, particularly for communication with Messages and Mail. But, I’ve set up VIPs in Mail to reduce these notifications to just the really important people.
I’ve been using this setup for a few months now and it certainly works for me. Even if this isn’t quite right for you, I’d encourage you to take a few minutes to really think through how you interact with your phone. I see far too many people with the default settings spending too much time scrolling around on their phones looking for the right app.
Like many of us, my online presence had become scattered across many sites: Twitter, Instagram, LinkedIn, Tumblr, and a close-to-defunct personal blog. So much of my content has been locked into proprietary services, each of which seemed like a good idea to start with.
Looking back at it now, I’m not happy with this and wanted to gather everything back into something that I could control. Micro.blog seems like a great home for this, as well described in this post from Manton Reece (micro.blog’s creator). So, I’ve consolidated almost everything here. All that’s left out is Facebook, which I may just leave alone.
By starting with micro.blog, I can selectively send content to other sites, while everything is still available from one source. I think this is a much better approach and I’m happy to be part of the open indie web again.
I’m very keen on backups. So many important things are digital now and, as a result, ephemeral. Fortunately you can duplicate digital assets, which makes backups helpful for preservation.
Most of my backup strategy was aimed at recovering from catastrophic loss, like a broken hard drive or stolen computer. I wasn’t sufficiently prepared for more subtle, corrosive loss of files. As a result, many videos of my kids' early years were lost. This was really hard to take, especially given that I thought I was so prepared with backups.
Fortunately, I found an old Mac Mini in a closet that had most of the missing files! This certainly wasn’t part of my backup strategy, but I’ll take it.
So, just a friendly reminder to make sure your backups are actually working as you expect. We all know this. But, please check.
My favourite spin studio has put on a fitness challenge for 2019. It has many components, one of which is improving your performance by 3% over six weeks. I’ve taken on the challenge and am now worried that I don’t know how reasonable this increase actually is. So, a perfect excuse to extract my metrics and perform some excessive analysis.
We start by importing a CSV file of my stats, taken from Torq’s website. We use readr for this and set the col_types in advance to specify the type for each column, as well as skip some extra columns with col_skip. I find doing this at the same time as the import, rather than later in the process, more direct and efficient. I also like the flexibility of the col_date import, where I can convert the source data (e.g., “Mon 11/02/19 - 6:10 AM”) into a format more useful for analysis (e.g., “2019-02-11”).
One last bit of clean up is to specify the instructors that have led 5 or more sessions. Later, we’ll try to identify an instructor-effect on my performance and only 5 of the 10 instructors have sufficient data for such an analysis. The forcats::fct_other function is really handy for this: it collapses several factor levels together into one “Other” level.
Lastly, I set the challenge_start_date variable for use later on.
## # A time tibble: 6 x 5
## # Index: Date
## Date Instructor avg_power max_power total_energy
##
## 1 2019-02-11 Justine 234 449 616
## 2 2019-02-08 George 221 707 577
## 3 2019-02-04 Justine 230 720 613
## 4 2019-02-01 George 220 609 566
## 5 2019-01-21 Justine 252 494 623
## 6 2019-01-18 George 227 808 590
To start, we just plot the change in average power over time. Given relatively high variability from class to class, we add a smoothed line to show the overall trend. We also mark the point where the challenge starts with a red line.
Overall, looks like I made some steady progress when I started, plateaued in the Summer (when I didn’t ride as often), and then started a slow, steady increase in the Fall. Unfortunately for me, it also looks like I started to plateau in my improvement just in time for the challenge.
We’ll start a more formal analysis by just testing to see what my average improvement is over time.
time_model <- lm(avg_power ~ Date, data = data)
summary(time_model)
##
## Call:
## lm(formula = avg_power ~ Date, data = data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -30.486 -11.356 -1.879 11.501 33.838
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2.032e+03 3.285e+02 -6.186 4.85e-08 ***
## Date 1.264e-01 1.847e-02 6.844 3.49e-09 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 15.65 on 64 degrees of freedom
## Multiple R-squared: 0.4226, Adjusted R-squared: 0.4136
## F-statistic: 46.85 on 1 and 64 DF, p-value: 3.485e-09
Based on this, we can see that my performance is improving over time and the R2 is decent. But, interpreting the coefficient for time isn’t entirely intuitive and isn’t really the point of the challenge. The question is: how much higher is my average power during the challenge than before? For that, we’ll set up a “dummy variable” based on the date of the class. Then we use dplyr to group by this variable and estimate the mean of the average power in both time periods.
data %<>%
dplyr::mutate(in_challenge = ifelse(Date > challenge_start_date, 1, 0))
(change_in_power <- data %>%
dplyr::group_by(in_challenge) %>%
dplyr::summarize(mean_avg_power = mean(avg_power)))
## # A tibble: 2 x 2
## in_challenge mean_avg_power
##
## 1 0 213.
## 2 1 231.
So, I’ve improved from an average power of 213 to 231 for a improvement of 8%. A great relief: I’ve exceeded the target!
Of course, having gone to all of this (excessive?) trouble, now I’m interested in seeing if the instructor leading the class has a significant impact on my results. I certainly feel as if some instructors are harder than others. But, is this supported by my actual results? As always, we start with a plot to get a visual sense of the data.
There’s a mix of confirmation and rejection of my expectations here:
Charlotte got me started and there’s a clear trend of increasing power as I figure out the bikes and gain fitness
George’s class is always fun, but my progress isn’t as high as I’d like (indicated by the shallower slope). This is likely my fault though: George’s rides are almost always Friday morning and I’m often tired by then and don’t push myself as hard as I could
Justine hasn’t led many of my classes, but my two best results are with her. That said, my last two rides with her haven’t been great
Marawan’s classes always feel really tough. He’s great at motivation and I always feel like I’ve pushed really hard in his classes. You can sort of see this with the relatively higher position of the best fit line for his classes. But, I also had one really poor class with him. This seems to coincide with some poor classes with both Tanis and George though. So, I was likely a bit sick then. Also, for Marawan, I’m certain I’ve had more classes with him (each one is memorably challenging), as he’s substituted for other instructors several times. Looks like Torq’s tracker doesn’t adjust the name of the instructor to match this substitution though
Tanis’ classes are usually tough too. She’s relentless about increasing resistance and looks like I was improving well with her
Other isn’t really worth describing in any detail, as it is a mix of 5 other instructors each with different styles.
Having eye-balled the charts. Let’s now look at a statistical model that considers just instructors.
Each of the named instructors has a significant, positive effect on my average power. However, the overall R2 is much less than the model that considered just time. So, our next step is to consider both together.
##
## Call:
## lm(formula = avg_power ~ Date + Instructor, data = data)
##
## Residuals:
## Min 1Q Median 3Q Max
## -31.689 -11.132 1.039 10.800 26.267
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -2332.4255 464.3103 -5.023 5.00e-06 ***
## Date 0.1431 0.0263 5.442 1.07e-06 ***
## InstructorGeorge -2.3415 7.5945 -0.308 0.759
## InstructorJustine 10.7751 8.9667 1.202 0.234
## InstructorMarawan 12.5927 8.9057 1.414 0.163
## InstructorTanis 3.3827 8.4714 0.399 0.691
## InstructorOther 12.3270 7.1521 1.724 0.090 .
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 15.12 on 59 degrees of freedom
## Multiple R-squared: 0.5034, Adjusted R-squared: 0.4529
## F-statistic: 9.969 on 6 and 59 DF, p-value: 1.396e-07
anova(time_instructor_model, time_model)
## Analysis of Variance Table
##
## Model 1: avg_power ~ Date + Instructor
## Model 2: avg_power ~ Date
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 59 13481
## 2 64 15675 -5 -2193.8 1.9202 0.1045
This model has the best explanatory power so far and increases the coefficient for time slightly. This suggests that controlling for instructor improves the time signal. However, when we add in time, none of the individual instructors are significantly different from each other. My interpretation of this is that my overall improvement over time is much more important than which particular instructor is leading the class. Nonetheless, the earlier analysis of the instructors gave me some insights that I can use to maximize the contribution each of them makes to my overall progress. When comparing these models with an ANOVA though, we find that there isn’t a significant difference between them.
The last model to try is to look for a time by instructor interaction.
## Analysis of Variance Table
##
## Model 1: avg_power ~ Date
## Model 2: avg_power ~ Date + Instructor + Date:Instructor
## Res.Df RSS Df Sum of Sq F Pr(>F)
## 1 64 15675
## 2 54 11920 10 3754.1 1.7006 0.1044
We can see that there are some significant interactions, meaning that the slope of improvement over time does differ by instructor. Before getting too excited though, an ANOVA shows that this model isn’t any better than the simple model of just time. There’s always a risk with trying to explain main effects (like time) with interactions. The story here is really that we need more data to tease out the impacts of the instructor by time effect.
The last point to make is that we’ve focused so far on the average power, since that’s the metric for this fitness challenge. There could be interesting interactions among average power, maximum power, RPMs, and total energy, each of which is available in these data. I’ll return to that analysis some other time.
In the interests of science, I’ll keep going to these classes, just so we can figure out what impact instructors have on performance. In the meantime, looks like I’ll succeed with at least this component of the fitness challenge and I have the stats to prove it.
This is a “behind the scenes” elaboration of the geospatial analysis in our recent post on evaluating our predictions for the 2018 mayoral election in Toronto. This was my first, serious use of the new sf package for geospatial analysis. I found the package much easier to use than some of my previous workflows for this sort of analysis, especially given its integration with the tidyverse.
We start by downloading the shapefile for voting locations from the City of Toronto’s Open Data portal and reading it with the read_sf function. Then, we pipe it to st_transform to set the appropriate projection for the data. In this case, this isn’t strictly necessary, since the shapefile is already in the right projection. But, I tend to do this for all shapefiles to avoid any oddities later.
## Simple feature collection with 1700 features and 13 fields## geometry type: POINT## dimension: XY## bbox: xmin: -79.61937 ymin: 43.59062 xmax: -79.12531 ymax: 43.83052## epsg (SRID): 4326## proj4string: +proj=longlat +datum=WGS84 +no_defs## # A tibble: 1,700 x 14## POINT_ID FEAT_CD FEAT_C_DSC PT_SHRT_CD PT_LNG_CD POINT_NAME VOTER_CNT## <dbl> <chr> <chr> <chr> <chr> <chr> <int>## 1 10190 P Primary 056 10056 <na> 37## 2 10064 P Primary 060 10060 <na> 532## 3 10999 S Secondary 058 10058 Malibu 661## 4 11342 P Primary 052 10052 <na> 1914## 5 10640 P Primary 047 10047 The Summit 956## 6 10487 S Secondary 061 04061 White Eag… 51## 7 11004 P Primary 063 04063 Holy Fami… 1510## 8 11357 P Primary 024 11024 Rosedale … 1697## 9 12044 P Primary 018 05018 Weston Pu… 1695## 10 11402 S Secondary 066 04066 Elm Grove… 93## # ... with 1,690 more rows, and 7 more variables: OBJECTID <dbl>,## # ADD_FULL <chr>, X <dbl>, Y <dbl>, LONGITUDE <dbl>, LATITUDE <dbl>,## # geometry <point>
The file has 1700 rows of data across 14 columns. The first 13 columns are data within the original shapefile. The last column is a list column that is added by sf and contains the geometry of the location. This specific design feature is what makes an sf object work really well with the rest of the tidyverse: the geographical details are just a column in the data frame. This makes the data much easier to work with than in other approaches, where the data is contained within an [@data](https://micro.blog/data) slot of an object.
Plotting the data is straightforward, since sf objects have a plot function. Here’s an example where we plot the number of voters (VOTER_CNT) at each location. If you squint just right, you can see the general outline of Toronto in these points.
What we want to do next is use the voting location data to aggregate the votes cast at each location into census tracts. This then allows us to associate census characteristics (like age and income) with the pattern of votes and develop our statistical relationships for predicting voter behaviour.
We’ll split this into several steps. The first is downloading and reading the census tract shapefile.
Now that we have it, all we really want are the census tracts in Toronto (the shapefile includes census tracts across Canada). We achieve this by intersecting the Toronto voting locations with the census tracts using standard R subsetting notation.
And, we can plot it to see how well the intersection worked. This time we’ll plot the CTUID, which is the unique identifier for each census tract. This doesn’t mean anything in this context, but adds some nice colour to the plot.
plot(to_census_tracts["CTUID"])
Now you can really see the shape of Toronto, as well as the size of each census tract.
Next we need to manipulate the voting data to get votes received by major candidates in the 2018 election. We take these data from the toVotes package and arbitrarily set the threshold for major candidates to receiving at least 100,000 votes. This yields our two main candidates: John Tory and Jennifer Keesmaat.
## # A tibble: 2 x 1## candidate ## <chr> ## 1 Keesmaat Jennifer## 2 Tory John
Given our goal of aggregating the votes received by each candidate into census tracts, we need a data frame that has each candidate in a separate column. We start by joining the major candidates table to the votes table. In this case, we also filter the votes to 2018, since John Tory has been a candidate in more than one election. Then we use the tidyr package to convert the table from long (with one candidate column) to wide (with a column for each candidate).
Our last step before finally aggregating to census tracts is to join the spread_votes table with the toronto_locations data. This requires pulling the ward and area identifiers from the PT_LNG_CD column of the toronto_locations data frame which we do with some stringr functions. While we’re at it, we also update the candidate names to just surnames.
Okay, we’re finally there. We have our census tract data in to_census_tracts and our voting data in to_geo_votes. We want to aggregate the votes into each census tract by summing the votes at each voting location within each census tract. We use the aggregate function for this.
ct_votes_wide <-aggregate(x = to_geo_votes,
by = to_census_tracts,
FUN = sum)
ct_votes_wide
As a last step, to tidy up, we now convert the wide table with a column for each candidate into a long table that has just one candidate column containing the name of the candidate.
Now that we have votes aggregated by census tract, we can add in many other attributes from the census data. We won’t do that here, since this post is already pretty long. But, we’ll end with a plot to show how easily sf integrates with ggplot2. This is a nice improvement from past workflows, when several steps were required. In the actual code for the retrospective analysis, I added some other plotting techniques, like cutting the response variable (votes) into equally spaced pieces and adding some more refined labels. Here, we’ll just produce a simple plot.
Distribution of the difference between the predicted and actual proportion of votes for all parties
Geographical distribution of the difference between the predicted and actual proportion of votes by Electoral District and party
