Menu Sign In Contact FAQ
Banner
Welcome to our forums

"Euroga Airports" database data accessibility

Dear Peter, dear all,

There is a topic that I’ve had on my mind for quite some time. It has been discussed already on several occasions outside of the EuEuroGA roga forum and I personally found it quite sad that a good/worthwhile topic has suffered (at least in two cases that I am aware of, one on a German forum and one in a Telegram group) from – depending on one’s point of view – offensive or at least non-optimal communication.

I would be grateful if we can have here on EuroGA itself a decent, hopefully objective discussion of the topic free from personal attacks etc, as should be and is the spirit of the EuroGA forum.

A lot of work and hours has gone into the EuroGA Airports Database project – as an IT consultant and occasional developer I know how painful that work can be. So first of all I want to say “Thank You” to Peter and David and to anyone who has contributed. The European GA community is not huge and everyone’s input counts to let it stay alive and thriving and it’s great if people step up and take the lead and add new ways of using the internet to help GA.

Today, there are still several such airport databases out there. There is a bit of a problem with reaching a critical mass. There is a German site which has a critical mass, but only in German and mostly used by German pilots, so the scope is limited. Also, it is technologically about a decade or two behind and the creator has stopped flying himself several years ago and is unlikely to invest much energy into a technology upgrade (although as far as I know, he still gets up early every single morning and curates and types in aviation news to his site – so kudos to that gentleman as well!). There are a few other sites out there, as well.

It is from my point of view the EuroGA airport database project that has pretty good chances of advancing to to become the “one-stop shop” for community feedback on European airports – based on a couple of factories, like the active community here on the site, the implicit European reach and the fact that there is a focus on high quality content and no commercial interests. So – a great starting point!

What has been noted several times now and is a real “show-stopper” in the eyes of some people is the ability to access the content not only in a human-readable format, but also in a machine-readable (i.e. “API”) format so that the community content can be re-used.

As introduced above, it would be great to have an open, respectful discussion on the topic and collect the pro’s and con’s.

Thank you!

Hungriger Wolf (EDHF), Germany

Yes indeed I got beaten up on the German domestic site (not realising they have their own – yet another old – airport database) a few times over this project

There are a number of reasons why I don’t want to do this right now.

One factor is that – like it or not – people contribute data according to their allegiance to a particular community. So for example pilots who like to support EuroGA tend to contribute to the EuroGA airport database. German pilots, especially if active on the German domestic site, tend to contribute to the German domestic airport database(s). And so on. This sort of thing is very hard to change. Impossible, I think, since most pilots are not 18 years old It’s like the old joke about four pilots on a desert island; a year later they have created four user groups, four forums, and each forum has 20 members When building a GA community one has to rise above this, and it is particularly challenging when doing it internationally. EuroGA has done it successfully.

Another one is that the cost of building and running the site is not in the data (which is contributed for free) but in the user interface, in the ongoing admin, and the hosting cost. Yet, it is the UI which enables people to enter the data! Around 10k was spent building the EuroGA UI; it is the work of several people (I can do only project specification and very basic admin myself) and I paid for that out of my pocket because I didn’t want to spend the annual EuroGA donations on it (they would have nowhere near covered it anyway). The hosting, done on a wholly separate server so that a hack of the airport database does not compromise the EuroGA forum and does not expose any personal data, is quite cheap at just £12/month. An API does absolutely nothing… zero, zilch. Somebody has still got to build a UI. And it has to be done well; there are nontrivial aspects e.g. image upload. It isn’t a quick PHP hack…

I can’t avoid commenting on the deep hypocrisy in the attacks I got. Since our database opened, I’ve had a number of requests to access the data. They seemed to come from hobbyists who were building some sort of aviation site. I replied along the lines that it is early days to think about that; we need to build the numbers up. And that really is true. If you are going to do a joint venture, it is always much better when you have something sizeable to offer. But I sent each enquirer to the main German site. The reply, in every case, was they they already tried that and got an immediate refusal! I would therefore ask (rhetorically only of course ) why did/does EuroGA get beaten up over this, in Germany, while their domestic databases refuse all access to their data?

Next, commitment. I would not expect anyone to contribute a single report, not even a single line, not even hit their keyboard, unless they are confident that the database is futureproofed, and – crucially these days, with thousands of attacks every day – has proper backups. Here on EuroGA we have a long term stable project, with proper funding, admin, daily backups (not stored on the same server of course), and with several trusted people having admin rights, if something happened to me, someone would pick it up.

Another factor is who would run this public-API database project. If you offer an API, and are fortunate enough to actually get someone else out there to build a UI for it, they will be demanding extra database features. These have to be added, which is easy (with database items), but they may be judged pointless, or time consuming to maintain, and who is going to be paying for the work? There would be a lot of haggling, and of course nobody in GA has any money! The only way to get money is to advertise and everybody hates that – as well as introducing difficult issues e.g. a disrespected maintenance company buying adverts and then suing if they get criticised. Somebody will ask for a database extension to show historical fuel prices, which is a hostage to fortune. Somebody will ask for opening hours, which is a far worse hostage to fortune, and is sure to produce some violent “feedback” when somebody tries to land at a closed airport. That is why we have the AIP links, which took a lot of work to get right.

Then you have security. Today is not 1995. Any site gets attacked and hacked if it can be hacked, destroyed if it can be destroyed, just for fun. Or covered with ads for fake passports… On the EuroGA airport database, we allow only registered EuroGA users to add reports. We use the robust OAUTH system for this, which doesn’t transfer any personal data. And the robust system for checking new signups works well with this. There is a Guest mode, added to save people time if they just want to add the odd report, and to enable people to contribute anonymously who would otherwise feel that contributing to a “EuroGA” database is disloyal to their domestic forum, and this is popular, but of necessity the report is invisible until approved by an admin. If you have an open API, how do you implement this? You would need a common “authorised user” database which everyone would have to agree to supplying and maintaining, or the server would need to perform multiple OAUTH queries (on different pilot forums) in succession… And nobody who runs a GA forum wants to open up any kind of data to other people who run a GA forum – see my joke above about 4 pilots That can also be solved but it is a fair bit of work and it all has to come together. In Europe, this has not happened and IMHO isn’t going to happen.

I am also quite busy and don’t need the extra work, or 5-digit expense, right now And this would be expensive. Bigger than all work done to date. There are many good people out there but not many work for free, and those that do tend to not do it for long. Then someone else has to pick up their work – and everybody hates doing that The EuroGA airport database got some great free coding contributions but those were well defined self contained projects; this sort of integration would be much bigger.

Finally, what is wrong with the EuroGA user interface? It was designed carefully, and has had CSS input from a specialist builder of mobile interfaces, so it runs on hopefully all modern platforms. If it does not, then certainly nobody has recently complained. It is basic in style and could be more flashy and colourful, but a lot of people hate “flashy” sites, especially with massive fonts, especially ones designed with “modern” frameworks. For example, a lot of users really hate the current fashion of coding a site so the Back button is disabled, but this (along with probably a thousand bugs) is what you get with modern “web artisan” frameworks. This makes me conclude that the calls for an API (this is not aimed at you Patrick ) which were forcefully aired on the German domestic forum, have been motivated by a desire to avoid the name “EuroGA” i.e. to avoid promoting EuroGA on the domestic territory. Well, there isn’t anything that can be done about this. An international forum does have these challenges and we have seen them play out here over the years. For the most part the discussion on EuroGA is above all that, and that has to remain the objective.

Maybe one day this can be looked at again, once the database is much bigger, and someone turns up who is really good at server side programming and is willing to work for free for some time. Right now almost nobody is flying…

Administrator
Shoreham EGKA, United Kingdom

Peter wrote:

One factor is that – like it or not – people contribute data according to their allegiance to a particular community. So for example pilots who like to support EuroGA tend to contribute to the EuroGA airport database.

Do you think that if the database would be accessible by other means additionally to through the particular website “airports.euroga.org”, it:

  • would discourage “people who like to support EuroGA” to contribute to it?
  • encourage people that access it through these other means to contribute to it?

I would think the latter.

Peter wrote:

Another one is that the cost of building and running the site is not in the data (which is contributed for free) but in the user interface, in the ongoing admin, and the hosting cost. Yet, it is the UI which enables people to enter the data!

Would making it accessible through other means by any way increase this cost? While you were the only one to contribute cash, were yourself, and the person (people?) you paid the only ones to contribute work and time?

Peter wrote:

An API does absolutely nothing… zero, zilch. Somebody has still got to build a UI.

An API allows alternate UIs to be built, like for example through a Telegram bot. Like, oh, I don’t know, the one I run, and which currently has only one unique, very niche, and very, very limited function? It would certainly make sense for querying the database. My gut reaction is that it wouldn’t be a good interface to enter a whole record (hmm… except maybe in a question-response kind of way?), but would be awesome for tidbits of information like “I just paid that much for fuel”, “I just paid that much for landing and that much for parking for that time”, etc. I certainly would contribute these tidbits more with such an easy interface. How these tidbits would integrate into the current structure that expects a full record at once is a question that still needs to be thought about.

Peter wrote:

I can’t avoid commenting on the deep hypocrisy in the attacks I got. Since our database opened, I’ve had a number of requests to access the data. (…) But I sent the enquirer to the main German site. The reply, in every case, was they they already tried that and got an immediate refusal! I would therefore ask (rhetorically only of course ) why did/does EuroGA get beaten up over this, in Germany, while the German databases refuse all access to their data?

The inquirers had hopes for EuroGA to be more open, and positive towards being a platform that “European GA” can build upon, and contribute to. In other words, EuroGA has/had a good reputation in this respect. The inquirers didn’t expect EuroGA to behave, on this point, as “bad” as the German database.

Peter wrote:

If you offer an API, and are fortunate enough to actually get someone else out there to build a UI for it, they will be demanding extra database features. These have to be added

I think “demand” is too strong a word, and so is “have to”. They would express different opinions than yours on the useful database features. Whether you are convinced by them or not, and add these database features or not, is another question. You can still exercise “editorial decision” on whether to implement database features or not, even if you offer an API.

Peter wrote:

Then you have security. (…) On the EuroGA airport database, we allow only registered EuroGA users to add reports. We use the robust OAUTH system for this, which doesn’t transfer any personal data. (…) If you have an open API, how do you implement this? You would need a common “authorised user” database which everyone would have to agree to supplying and maintaining, or the server would need to perform multiple OAUTH queries (on different pilot forums) in succession…

The different UI would need to authenticate with OAuth to the API, as the final user’s identity, not a “common authorised user”. I also fail to see where the “multiple OAuth queries in succession” would come from. I’m not sure whether in your mind the Airports database would open up to other OAuth providers or not.

Peter wrote:

I am also quite busy and don’t need the extra work, or 5-digit expense, right now

No contest to that.

Peter wrote:

Finally, what is wrong with the EuroGA user interface?

It is a “one size fits all” approach. Imagine that all email users would have to use the same UI, just to communicate, instead of using one from any number of programs and UIs to choose from. That would be a worse world. (Which is also a reason why in my dreams, I wouldn’t be forced to the EuroGA website to read/write on the EuroGA “forum”; I would love for it to be something more like a … Usenet newsgroup / email mailing list / Fidonet echo area that one can use from any UI one wants, including a “canonical” “forum UI” website for the users that like that.)

ELLX

I would think the latter.

Certainly – but who is going to pay for the development?

Would making it accessible through other means by any way increase this cost?

It would generate a lot of cost during the integration. There is a reason why certain other sites out there look so old: the are old. They were done in the early days of PHP + DB. It is likely that their proprietors would have a hill to climb to do new code, or the original programmer is long gone.

While you were the only one to contribute cash, were yourself, and the person (people?) you paid the only ones to contribute work and time?

The EuroGA database was probably 90% built by a Polish programmer, at a few hundred € per day, with the other 10% done by volunteers (such as yourself) doing specific parts.

The inquirers had hopes for EuroGA to be more open, and positive towards being a platform that “European GA” can build upon, and contribute to. In other words, EuroGA has/had a good reputation in this respect. The inquirers didn’t expect EuroGA to behave, on this point, as “bad” as the German database.

That would be nice, but definitely not what was written

You can still exercise “editorial decision” on whether to implement database features or not, even if you offer an API.

Indeed, but the others would quite likely choose to walk away. Some databases contain a lot of extra stuff – even if most of it is useless / obsolete. This is a challenge to present on a mobile device; basically, you can’t. On a PC, which is all people had 20 years ago, you can do anything, and you can do a rich UI, beautifully.

The different UI would need to authenticate with OAuth to the API, as the final user’s identity, not a “common authorised user”. I also fail to see where the “multiple OAuth queries in succession” would come from. I’m not sure whether in your mind the Airports database would open up to other OAuth providers or not.

You have to authenticate against “something”. Currently, we use OAUTH to check that the contributor (who is trying to log in) has a EuroGA forum profile and it is approved for posting. One would need some sort of similar mechanism to work against other user databases.

One could not have people able to contribute without authenticating them against “something” which has previously done some sort of verification on them. Well, other than a “guest” process, like we have, but that implies

  • having 1 or more human authorisers
  • sending a notification (probably an email) to all of them upon each Guest record creation
  • develop a UI for shared admin on which each admin can authorise a Guest report
  • hoping that at least 1 of them has the time to check each new Guest record and click on the Make Visible button if it looks ok (I used to do this for the Autorouter for its first year or so, approving just about every signup except those with an email like [email protected] )

Also, Guest reports are not ideal because they obviously cannot be edited after the initial Save. Well, unless one makes some assumptions e.g. same cookies, which will backfire as soon as someone wants to edit the report when they got home and get onto another device.

I would love for it to be something more like a … Usenet newsgroup / email mailing list / Fidonet echo area that one can use from any UI one wants, including a “canonical” “forum UI” website for the users that like that.)

Unfortunately, Usenet etc are all (practically speaking and with some exceptions) long dead.

It takes something like 50hrs of a good programmer’s time just to develop a very basic UI for a database like this. You ideally need a second person to do testing, plus more testing of client devices to sort out the CSS (just using bootstrap, as we do on both EuroGA sites, is not a complete solution ). These projects always take much longer than one would think.

Not wishing to sound overly negative but I think it will be better to review this later in 2021 once the database is much bigger and much busier. Right now, as I said, almost nobody is going anywhere.

I am “all ears” re improvements and one which was discussed was a telegram interface. The challenge there is (a) how to authenticate and (b) how to present a UI for adding a record. It would be easy enough to do a read-only UI, especially for wx, notams and the AIPs. I already have most of these on a private site.

We could also improve the picture handling, by using a library like Dropzone. I don’t like these in general because open source code is great until the programmer gets himself a girlfriend and after that you have to pay somebody to unravel thousands of lines of code to fix something, and they make hacking much easier (every PHP forum has been hacked and trashed, ofen multiple times) but DZ has a wide user base and is under active development – for now at least.

Administrator
Shoreham EGKA, United Kingdom

Peter, Thank you for taking the time to write up these detailed responses.

For some of them, I have a very different perception and let me try to address these.

Hypocrisy: I’m sorry, Peter, but here I’m sincerely convinced that the misunderstanding is much deeper than the hypocrisy. As @lionel has pointed out (and also, frankly, I have tried to point out on the other forum), people approaching you with this proposal is NOT an offense. It is a compliment (even by those who pick the wrong tone) and it is a sign of trust in the momentum of this project. Nothing more, and nothing less. It’s a shame, really, to block this off.

Imagine someone approaches Jimmy Wales, the founder of Wikipedia, with an idea how to improve the Wikipedia project. Imagine he responds: “Well, the hipocricy! Why don’t you all go to Encyclopedia Britannica! Or Brockhaus! Why don’t you ask them?” — “Well, Jimmy, we ask you because we believe in your project and we have – say – our doubts that Encyclopedia Britannica is the future…”

It is similar here. The European GA community needs a European solution to this topic. That is NOT going to be a domestic country site (regardless if it has an API or not). I, for one, personally strongly believe we need a European solution and it would be great to have it – and my best candidate is the very one you’re building. Again, it is a compliment based on everything you have built already and I fail to see why this is offensive.

The same is true for when you mention that people will demand more features. Well, yes – of course! If people believe in the project and it becomes bigger, people are going to have ideas! Again, that is not offensive! It is a sign that the project is alive. At work, I’m fighting with software products where there is no user input. The projects are dead, no one cares, no one puts any passion into it. That is horrible. People posting ideas is great. That does NOT mean that one person (or anyone, for that matter) is “expected” to implement everything by nightfall. But it is a blessing to have a backlog full of ideas, not a curse.

Next: This is NOT about saying the current UI is bad. No-one said that! I’m not saying Google’s UI is bad, but how successful would Google be if they kept their data in silos? The only one in the consumer industry who can really get away with being a d*** about access to their ecosystem is Apple and they get (and deserve) a ton of criticism for that. For example, I can easily feed my Microsoft 365 To Do items into the Apple Reminder app. I do that – not because the Microsoft UI is bad (it isn’t) – but because I want all my To Do’s in one place. Apple does not allow this the other way around and I wish there’d be a time where they can’t get away with that arrogance anymore. It just doesn’t fit the 2020s – the rest of the world has moved on beyond that kind of thinking. But we’re digressing.

Now, regarding the effort to do what I’m proposing (and what others are proposing) I firmly believe you are grossly overestimating it. You are mentioning 10.000s… I do not believe that. I think it is much less effort.

If we are looking at a read-only access only (and that is a HUGE step forward and what most people refer to when they mention access to the user contributed data), there are currently two php pages serving this:

These two php scripts do the following:

  • Interpret the query parameters (such as: icao = EGKA)
  • Look up the database with those query parameters
  • Render the result of the database query as an HTML page

The only thing that people are asking as a feature to be added at the moment would be to allow those two pages to render the output in one additional format, i.e. allow rendering in JSON in addition to HTML. It is so easily done, we are spending more time debating it.

E.g. the URL could be: https://airports.euroga.org/search.php?icao=EGKA&format=json

and the result could be a simple JSON string such as:

“13”: [{
“icao”: “EGKA”,
“runway_condition”: “hard/good”,
“taxiway_condition”: "hard/good ",
“ramp_condition”: “hard/good”,
__ “landing_fee”: “30.00 GBP”,__
“…”: “…”
}

You are saying this is bigger work than anything that has been done already on this project. I say: The contrary is the case. Everything is already there. The only thing needed is adding the ability to expose the same data in a slightly different output format.

No user authentication required, as you do not required users to authenticate currently when they view data.

This alone would so extremely useful to anyone building great things on top if it! I already have ideas that sparkle my imagination and hey – maybe you’ll like to use the outcome of that what is currently imagination only yourself as well. And I can imagine other people having other, probably better ideas!

Now, I have written a lot of text, but let me offer one thing:

If you allow me to build that functionality as a “proof-of-concept”, e.g. on a test environment, I will happily do that. For free, obviously. You then run that proof-of-concept code by whoever might want to check it if it makes sense and if everyone is happy, maybe, just maybe, it shows itself as a useful addition to the project without the dreaded expenses?

Hungriger Wolf (EDHF), Germany

OK; nothing actually suggested was taken as “offensive”. I would not react like that unless it was clearly intended.

My only concern, if someone suggested an API and didn’t make insinuations about motives etc etc, is how much work/expense it might be for me, and other downstream issues such as integration with other sites (for example David spent ages integrating EuroGA and the Autorouter login, and that was just some OAUTH related code).

If you think a read-only API is worth doing, that as you say is easy enough. Actually anybody can do that now; it’s just a bit tacky to scrape a website (although I note a server running in Germany is diligently copying the database already ).

Let me pass this by the Polish guy and see what he thinks. It’s probably better for me to try him first because the did the original code.

However, all that a read-only API will achieve is making it easier for someone to write a different UI. Would that increase the number of reports being filed?

Administrator
Shoreham EGKA, United Kingdom

Who benefits from read only access?

From what I can see, it would be whatever website / app was getting the data. I’m not sure that benefits EuroGA in any way.

Of course EuroGA could require a credit beside any data. Not sure if that is worth it or not.

EIWT Weston, Ireland

I don’t get it.

There is a simple technology that enables any other site to make the airport database available. It’s called a link.

Single sign-on would be nice, but is hardly essential.

Biggin Hill

Cobalt wrote:

There is a simple technology that enables any other site to make the airport database available. It’s called a link.

OP suggested API so the contributors to other sites can access EuroGA database via their GUI both read and write. Using this approach such users would be unaware of EuroGA and would access airport data “under the hood” accessing only their original site. By using link these users would first have to become EuroGA members prior to being able to access database. And it seems that this idea is not well accepted by the moderators of other sites.

LDZA LDVA, Croatia

A read-only API in the form described above is simple to implement and serves one fundamental goal: It will allow the reach of the project to extend beyond its own limitations.

Respectfully, the notion mentioned above is that it’s just about someone hacking another UI on top of it and why would anyone need that, with a working UI in place? One poster even suggested that a hyperlink will do the trick.

Try to think this a bit bigger: From my point of view, the success of such a project depends on three core ingredients:

  • An active community
  • Quality content
  • Reach

The EuroGA Airport Database project benefits from an active community around it and the way it has been setup encourages quality content, so we can tick 2 out of three of the boxes.

Now, how about reach? I firmly believe that the more open we are about the content, the more we encourage other people, other services, other developers to build upon that quality community content, the more the community of active contributors will grow – it feeds back into itself! Say that out of 10 people who consume the content provided, one will become an active contributor (the number is arbitrary and could be any number), then extending the “read-only” reach will also grow the content, making it again more attractive for everyone.

I’m advocating the idea of embracing ANYONE to use and further promote the data to eventually become the one-stop shop for European airfield reviews. By no means is the proposal to do this necessarily in a “secretive” way as described by Emir.

To give an example of how I would like to use the data (and yes, besides my generic hope for the database to become the one-stop shop as described, it is this idea of mine that triggers me to write so passionately about this topic here ): I’m working on an idea to provide aircraft-specific briefing and other information in a (among others) chat bot interface that can again be consumed via e.g. Telegram, Alexa, and other messenger applications but might also be consumed via a graphical Web UI. So basically you could talk to the virtual alter ego of “your” airplane and get information about e.g. a landing at a particular airfield. That includes several data points such as weather/wind data (available via APIs), field elevation (available via the Google Elevation API), runway dimensions and aircraft specific landing distance factors as per POH. This covers the “can I land there” aspect, but it would be great if one could add the “do I WANT to even go there” aspect and that can only be done based on reviews by other airport users. Or I’d love to ask the bot to send me the reviews for airfields in a 25 NM radius of where I just landed. No @Cobalt, this cannot be implemented with “a link”. This would require me to query the Google API (for example) to find the ICAO codes of airports within that radius and take the JSON result of that query to the EuroGA API and query the reports for those ICAO codes, to then provide a nicely formatted answer to the user.

Another potential use case is apps that have a different focus (like RunwayMap or several of the VFR flight planning tools), which again run their own little airport review databases. How great would it be if all those would source the same data, like so many services are sourcing wikipedia data?

It is the argument brought forward on the German forum about this as well: Me, as a user, I’m not willing to contribute my data for each airfield I have visited to the eddh PIREPS, to the EuroGA database, to the SkyDemon airfield report, to the RunwayMap app, and, and, and…. I’d feel foolish to do so. The maximum a user will do is contribute to ONE of these. And to become THE one that people contribute to, you need to build and maintain a critical mass/reach and preferably BECOME the database underlying any of these mentioned above! I, personally, would want to contribute to the ONE project that allows all others to use its user provided content rather than being a closed-off silo – that’s how it should be and that’s how it’s done for any other successful user-generated, non-commercial data collection that didn’t get stuck in 2010s thinking IMHO.

Hungriger Wolf (EDHF), Germany
33 Posts
Sign in to add your message

Back to Top