Follow, retweet @dailypinster




Post new topic Reply to topic  [ 162 posts ]  Go to page 1, 2, 3, 4, 5 ... 11  Next
Author Message
 Post subject: The Pretty Charts thread
PostPosted: Fri Feb 12, 2016 5:30 am 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
Calling all Pinsters! I'm looking for anyone willing to contribute their very useful data, currently floating around the Pin in miscellaneous forms. I'm playing with some ideas for prettifying it. You would still "own" the data and get to update it whenever you want, according to your own schedule.

To kick the tyres of it, I started with metalmike's PPR numbers from here. (h/t to metalmike for the collaboration so far). The object was to capture the monthly and quarterly numbers, year-on-year changes, and so on, all in one compact chart instead of the "wall of text".

Here is the result of the initial tyre-kicking: note the numbers are not real (yet). Click on the screenshot below to go to the real interactive version in a new window/tab. I need to discuss further with metalmike before this is finalised.

Image

The charts are in beautifully resizable SVG -- no more fugly distorted bitmaps. The underlying technology is the wondrous D3.js library. The actual data is in simple csv or tsv form like this (I was hoping it could also be edited in Google sheets, but this hasn't worked out [yet]):

Code:
Xlabel,2013,2014,2015,2016
Jan, 400, 603, 1254, 125
Feb, 400, 603, 1254, 125
Mar, 1100, 1200, 1300, 0
Apr, 1255, 1807, 1042, 0
May, 1161, 1149, 1035, 0
Jun, 1272, 2277, 1764, 0
Jul, 1255, 1807, 1042, 0
Aug, 1161, 1149, 1035, 0
Sep, 1272, 2277, 1764, 0
Oct, 1255, 1807, 1042, 0
Nov, 1161, 1149, 1035, 0
Dec, 1272, 2277, 1764, 0


Volunteer data contributors would be able to keep the data updated without having to worry about how the chart is done, while hard-core hackers can contribute their hand-coded charts. Here's the code for the bar chart above. Bar charts are only the beginning. D3 allows data to be attached to any SVG images. Here's one borrowed from Wikimedia Commons and tidied up with Inkscape:

Image

Once in the right form it's a (relatively) simple matter to tag the counties with data, and create heat maps, chloropleths etc. I'm hoping to do this with my own price/sq.ft. data.

Ideally we could keep some kind of structured list of pretty chart links, highlighting which ones had been updated recently. Anyone interested in prettifying their existing data or charts? Doesn't matter whether you are planning to hack it yourself, or just contribute data and let someone else design the visuals. Reply here or PM me.

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Fri Feb 12, 2016 6:00 am 
Offline
Nationalised
User avatar

Joined: Nov 6, 2006
Posts: 8542
Location: Australia
are you able to grab the gps location data from daft for houses along with the price drop data and price itself.
Be interested to interpolate the price drop data and the average price data. Classified by apartment/house/site/number of bedrooms/ sq footage/ or a combination of each.

If you want someone to do that I can certainly help :) I`ve also done some work with open layers.


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Fri Feb 12, 2016 6:08 am 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
Blindjustice BATONEFFECT wrote:
are you able to grab the gps location data from daft for houses along with the price drop data and price itself.
Be interested to interpolate the price drop data and the average price data. Classified by apartment/house/site/number of bedrooms/ sq footage/ or a combination of each.

If you want someone to do that I can certainly help :) I`ve also done some work with open layers.

Interesting idea. For now, though, I'm not looking to do new data sources or scrapes, just to collate and prettify data that people are already producing on the Pin on a regular basis. Indeed, some people are already producing their own pretty charts. This is just an idea-starter for those that aren't. For new data, I'd say go and produce/scrape it first, then think about how it might be presented visually.

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Fri Feb 12, 2016 7:22 am 
Offline
Nationalised

Joined: May 13, 2008
Posts: 11442
Location: Somewhere up in the hills
Very interesting.


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Fri Feb 12, 2016 12:28 pm 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
Gonna have a stab at creating an Ireland map example today. Code will be updated every now and again at:

PinsterDev on GitHub Gist

I'll keep a mini-project diary there for the people who said they were interested in hacking.
bl.ocks.org is a cheap place to host our running code. You can see it here:

Map example on bl.ocks.org

When something is running there'll be a link there to view in separate window mode (without the code/supporting files).

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Last edited by ps200306 on Fri Feb 12, 2016 3:44 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Fri Feb 12, 2016 12:46 pm 
Offline
Of Systemic Importance

Joined: Nov 4, 2011
Posts: 5502
Location: SthDub
Fair play PS, great work


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Sat Feb 13, 2016 10:41 pm 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
Eureka! A snapshot of asking prices in the first quarter of 2011 ... I'll spruce this up and stick up the singing, dancing, animated version in a day or two.

Image

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Wed Feb 17, 2016 5:46 pm 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
Ok, here is version 0.1 ... I need to fill in values and tooltips on the maps plus various tidy up. You can see values and YoY changes on the table view for now. The animation of the Ireland map gives you an idea of how things have centred on Dublin since 2012, even if only with colours. You should be seeing something like this when you click the link ... I have only tested on Firefox so far. In of a bit of a rush, so wanted to get a version out there. Comments, suggestions and bug reports welcome.

Image

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Wed Feb 17, 2016 5:56 pm 
Offline
Planning Tribunal Attendee

Joined: Apr 21, 2008
Posts: 1460
How's the raw data generated - the price per square foot/county and post code?

I've done a bit of work on mapping stuff using the R and the shape files from the CSO
this on population statistics for instance:

Image

I'd love to play around a bit with property price data, but depends on the data!


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Wed Feb 17, 2016 6:54 pm 
Offline
Too Big to Fail

Joined: Sep 13, 2012
Posts: 4752
ps200306 wrote:
Ok, here is version 0.1 ... I need to fill in values and tooltips on the maps plus various tidy up. You can see values and YoY changes on the table view for now. The animation of the Ireland map gives you an idea of how things have centred on Dublin since 2012, even if only with colours. You should be seeing something like this when you click the link ... I have only tested on Firefox so far. In of a bit of a rush, so wanted to get a version out there. Comments, suggestions and bug reports welcome.

Great work, very interesting.

It doesn't work in Chrome. :-)

_________________
"It's easy to confuse what is with what ought to be, especially when what is has worked out in your favour"
Tyrion Lannister


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Wed Feb 17, 2016 11:34 pm 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
Eschatologist wrote:
It doesn't work in Chrome. :-)

Or Safari on iPad, at least not on the train :D
Will get to it shortly.

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Thu Feb 18, 2016 3:02 am 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
jess wrote:
How's the raw data generated - the price per square foot/county and post code?

It's webscraped quarterly from myhome.ie.

jess wrote:
I'd love to play around a bit with property price data, but depends on the data!

Here's the data and the current code.

https://gist.github.com/pinsterdev/898435e2f8561941b628

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Thu Feb 18, 2016 3:13 am 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
Eschatologist wrote:
It doesn't work in Chrome. :-)

Not sure if I should be relieved, but it seems to break in the same way on Chrome, Safari and Edge. I presume the table bit worked? -- it does for me in all three. Hopefully the maps are just a small scaling problem (famous last words). Will look at that, plus adding text to maps, tomorrow.

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Thu Feb 18, 2016 8:48 am 
Offline
Planning Tribunal Attendee

Joined: Apr 21, 2008
Posts: 1460
ps200306 wrote:
jess wrote:
How's the raw data generated - the price per square foot/county and post code?

It's webscraped quarterly from myhome.ie.

jess wrote:
I'd love to play around a bit with property price data, but depends on the data!

Here's the data and the current code.

https://gist.github.com/pinsterdev/898435e2f8561941b628


Thanks.

I think I've found a chunk of code that allows me to pull geo-coordinates from google using an address.
If I can get it to work I can run the PPR against that, then use the CSO maps to plot the data based on either small areas or electoral areas.
If I do it for Dublin it might give a clearer indication of what's happening there.
Such a pity there isn't decent size information in the PPR.

There goes today's productivity.


Top
 Profile  
 
 Post subject: Re: The Pretty Charts thread
PostPosted: Thu Feb 18, 2016 2:35 pm 
Offline
Nationalised
User avatar

Joined: Apr 1, 2010
Posts: 9773
jess wrote:
I think I've found a chunk of code that allows me to pull geo-coordinates from google using an address.
If I can get it to work I can run the PPR against that, then use the CSO maps to plot the data based on either small areas or electoral areas.
If I do it for Dublin it might give a clearer indication of what's happening there.
Such a pity there isn't decent size information in the PPR.

There goes today's productivity.

That sounds neat. Interested to see the results. I tried using Google APIs a while back to get coordinates from PPR addresses but free usage was limited to a few hundred translations per day, and I ran out of patience. Hope you don't hit the same limits.

_________________
"Prediction is very difficult, especially about the future" – Niels Bohr


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 162 posts ]  Go to page 1, 2, 3, 4, 5 ... 11  Next


Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Jump to:  

Follow, Retweet @dailypinster



Pyramid Built, Is Better Built! - Latest Property Discussions www.thepropertypin.com