3 for 3, baby!
As in, my two data patches and code patch all received business approval to get tossed into production today. My weekend was spent watching our handwritten replication process chugg along after fixing a pretty large dataset that managed to fall between the cracks of our sync procedures.
Today is going to be spent fixing datasets that were never generated properly due to a variety of causes, mostly the past developers not finishing their work. I've learned that creating a button that says one thing while having nothing in the application that does anything remotely to the description will cause the client to believe that the functionality existed at some point. Of course no body can honestly remember if that feature was promised, or how it was even supposed to work; but its a priority bug now so it needs to do something.
My side project is coming along well. I've recently rekindled an interest in my city's crime statistics and came the realization that our interactive crime map left some things to be desired. Specifically heat maps, graphs (actually any historical data past a few months; it seems to only show recent datasets) and intelligible search functionality. I've written a Excel loader and dump all of the city of Houston's crime data into a SQL database. I plan on having a really simple web application to show the results correlated on a google map using gmaps.js.
The loading process was a pain. Some workbook's schema was completely different so I had to tweak my loading sproc and engine for a handful of datasets and revert back. I found solace in the fact I could curse the name of the person who created the excel file due to the metadata.
So Hubski, what are you working on today?
I'm taking full ownership of some research data, and today I'm creating a new, less disorganized schema (Excel file lol) for tracking patient enrollment, testing, results, followups and data entry. The system we are using now is a patchwork creature that Mary Shelley would recoil from in terror. It started as a template created by some Ivy-leaguers who would never actually participate in the project I'm working on, and quickly ballooned as research coordinators (Like myself) found that they needed to keep track of more and more things, which has resulted in a document that is really really time consuming to use, and never looks good. Unless of course someone up in clinic decides they want to participate, in which case that whole plan get's thrown out the window.
regular non-fun job punctuated by the discussion of a fun code project that I am working on. The code takes public broadcast aircraft position data from ADS-B and does some heuristic analysis of the pilots' handling of the aircraft.
As someone who used to work with Transponders, I'd be interested in hearing more about what kind of analysis you are doing. Im familiar with using an SDR to receive ADS-B data, but hadn't heard of much analysis beyond graphing of current position, altitude, speed, etc. Unless you are near an airport, chances are most broadcasts that are strong enough for you to receive are going to be from passenger jets with high power Transponders, and unfortunately when planes are at cruising altitude, they tend to follow pretty standard flight plans, barring weather conditions, etc. If you could get a receiver near a small airport in an affluent area (where GA aircraft have been fitted with ADS-B OUT already), then I think you would hear much more interesting things :). Are you listening to the 978 MHz band, or just the 1090 Mhz band?
The Asiana incident is the genesis for the project. If you go to the pprune forums, there is a lot of discussion of approach profiles related to this accident. Through some cute analysis, you can come up with scaled metrics on approach and departure quality. From there, you can do a statistical analysis. So I have a 1090 recorder parked at the local airport recording and sending to a database for further analysis. There is a mix of traffic, the majority of which have ADS-B out. Filtering the data is the tricky bit...
Making a separate reply, but if you are interested in mapping approach paths, Mode C and Mode S (non-ADS-B) data on the 1090 band may still have interest for you. It will be difficult to correlate, however Mode-S contains the ICAO address and altitude info (even at its most basic without GPS), which you could at least map to Flight IDs and the filed flight plan, giving you a basic approach path in the X and Y direction, which you can add semi accurate (100 or 25 foot resolution) information in the Z direction. Even harder to correlate, Mode C data could be used for non-correlatable, but still trackable info to watch a plane land (gather the data, if a Mode-C target went from in-air (>N feet above airport MSL corrected for pressure altitude) to on-ground (<N feet above airport MSL correct for pressure altitude), you could still get approach information for almost any aircraft, not just ADS-B aircraft. Note: Mode C data has 100 foot resolution, and is pretty noisy.
That is ultimately the goal. Having a ubiquitous metric for evaluating pilot performance when subject to different airspace/approaches/airport environments. Right now, I would be happy with reliable interpolation of spacial data. The broadcast packets are temporally inconsistent and the speed/altitude/heading are spread over more than one packet. So a simple interpolation only goes so far. I expect to have to write code to intelligently interpolate based on what aircraft can actually do. So no 3g turns in a transport airplane in the traffic pattern, etc. Once I have a functional model with the "tidy" ADS-B data, then the challenge will be making the process robust enough for mode S/C. It's a plan subject to change...
Very interesting. I guess I had supposed that most pilots on instrument approach would follow the prescribed approach cards, though I guess that Asiana flight really proves that isn't always the case, as well as inclement weather, etc. You mention that most traffic has ADS-B out. Are you listening to non ADS-B Mode S transmissions, or any logging of Mode A/C transmissions? From a single monitoring point, I know that Mode A/C/S transmissions are far less interesting than ADS-B, however I was wonder where your "majority" claim came from. I suppose if you are near a commercial/passenger airport (as opposed to private/GA airport), this will probably be true. I'd be happy to support your efforts if theres anything I can do (I come from an Embedded/Avionics background, including TCAS/Transponder development, as well as some data processing experience), though I can help more with Avionics theory than I can with statistics :). Feel free to send me a PM if you would like to discuss. Best of luck to your efforts, it definitely seems like a cool project!
I am on training calls, facilitating a "welcome" video for a startup and making a list of new #sellingwithtng ideas. And I'm going to send some protools sessions to a musician I'm working with on a new album. Tonight I plan on watching Jonathan Strange and Mr. Norrel. Busy!!
Sounds nice. I'm researching oligodendrogliomas and any recent treatment advances. It's the form of brain cancer I'm surviving. I have a ride in an MRI tomorrow to ensure that my post-treatment disease has not returned. I get an MRI check every nine months currently. Going tomorrow is akin to a visit to the dentist. A visit to the dentist that may reveal a cavity that will slowly kill me. I'm a bit nervous.
I'm sorry you have to have that sort of day tomorrow or any other day. That's a heavy weight to carry with you. I'm sure others will join me in wishing you the best possible results tomorrow. Do you get the results quickly or do you have to wait for a radiologist to view the scans? Good luck!
I wanted to wish you well this morning and didn't quite know how to phrase my well-wishes. "Go get them!" Or "you can do it," just don't seem appropriate. How about, "have an awesome day."? -nah, that doesn't quite convey the intention either. Well, I'm sure I speak for others on Hubski when I say we are pulling for positive results. Does your school year begin again soon? Did you run in that race up north with your bro? If so, who won? :)
Thanks for the well-wishes. That's plenty. I've not yet seen a "Have a great post-treatment brain cancer MRI follow-up." card anywhere yet. By the way, the results came back the way we wanted. No change. Everything is stable and no evidence of disease progression. Great news. I get my next check in 12 months. My students return on Sept 7. The state of Michigan has set things so kids start after Labor Day. I'm moving from one middle school to the other, so I've been in the last 5 weekdays moving materials. I'll be setting up for at least another few days too. mk didn't tell you about the race? Our team came in 5th in the inaugural SISU Challenge. Not too bad. It's likely we'd had a better result if not for some "bottle-neck" traffic jams at a few of the race obstacles where we lost several minutes or more.
Altogether it was pretty awesome. We're planning on doing it again next year.
That's awesome! Congrats on the good news! That's really great. I forgot, he did mention the race on Hubski. I remember because he too cited the bottleneck as the reason you didn't place higher. Good luck with the new school year in the new school. I hope it's a welcomed change of scenery.
Taking dashnhammit's advice and I'm working on outlining a 1, 2, 5, and 10 year plan of where I would like to be, what I would like to have accomplished, and how I think I can go about accomplishing those personal and business related goals. Might PM a few of ya about it once it's done to see how completely unreasonable it is.
I've been learning about A/B testing, and am using it to test the color scheme on my website. I've always struggled with the design aspect, and I hope A/B testing will allow me to objectively improve the design.
I am currently working through Create Your Own Programming Language. I took compilers in Uni, but I want to brush up on some things. I would like to eventually create a transpiler to convert Python -> Go.My side project is coming along well. I've recently rekindled an interest in my city's crime statistics and came the realization that our interactive crime map left some things to be desired
That map is pretty is awful. I can understand why you would to create something better I plan on having a really simple web application to show the results correlated on a google map using gmaps.js.
Sounds neat. I would be interested in seeing the source once you have something going.
Drafting a Shareholders Agreement mostly. Talking with a client that is having problems with a major client of theirs. Trying to get a Share Purchase Agreement going based on a Memo of Understanding that I drafted last week.
For anonymity's sake I won't say the exact title, but it's about phrase processing. I'm at the stage of actually designing the poster so it's less about writing and more about arranging information in a way that makes sense visually. I'm terrible with design so it's a nightmare :( In general though, my advice for writing is to assume nothing of your audience. You've been thinking about these ideas for months or years and you have your own way of talking about it that other people likely won't understand. Make sure labmates and friends are able to understand the logic and follow what's happening!
I concur. By appearances, there are a good number of folks here familiar with the academic process. #peerreview?