Menu Sign In Contact FAQ
Banner
Welcome to our forums

Further evidence of poor theoretical knowledge tests

This from the BGA:

Since end 2020, inspectors have been required to hold a Part66L licence to sign a release to service for a Part 21 aircraft (most gliders are Part 21 aircraft). As you may recall, BGA arranged a process at no cost to inspectors to ‘grandfather’ their experience into Part66L licences by means of a simple application.

At the end of 2020, there were around 360 BGA inspectors. Natural turnover is around 10 inspector each year. Between 2020 and 2022, the CAA’s slow development of Part66L theoretical knowledge testiung meant that no new Part66L licenced BGA inspectors were qualified in that period. The CAA’s theoretical knowledge Part66L exams online testing system has been live since 2022. However, we have discovered that quite a few of the questions it uses are not fit for purpose. That has resulted in a lack of confidence in the CAA’s theoretical knowledge testing system. As you’d expect, we are proactively responding to the problem which, as it involves CAA and a CAA process, will not be resolved in the timescale we’d like and expect.

It’s not just pilot testing (including the “airspace bust” test that Peter highlighted a while back), the whole theoretical knowledge testing system appears to be rotten. (It seems I was very fortunate to be grandfathered in, otherwise our club wouldn’t have a glider inspector or any prospect of one for some time – problematic given we’re on an island and it’s expensive to fly one in from elsewhere).

Last Edited by alioth at 11 Apr 08:27
Andreas IOM

alioth wrote:

It’s not just pilot testing (including the “airspace bust” test that Peter highlighted a while back), the whole theoretical knowledge testing system appears to be rotten. (It seems I was very fortunate to be grandfathered in, otherwise our club wouldn’t have a glider inspector or any prospect of one for some time – problematic given we’re on an island and it’s expensive to fly one in from elsewhere).

What can you expect when the questions are (supposedly) secret? Both from a quality and learning point of view, this is just crazy.

ESKC (Uppsala/Sundbro), Sweden

Is there a precedent or legal framework for the BGA to partially secede or devolve from the CAA? In order to create their own glider-only Part 66 examination system.

If it’s an online system, it should be easy enough to monitor the questions while a certain number of candidates take the exam, then a freedom of information request for particular questions to get them officially. Changing them is a different matter. Even in France, with a published question bank, it took maybe 2 years to change a bad question. No-one likes admitting they’re wrong, especially if in anonymous unaccountable authority 😟

If it’s any consolation, Mike Busch had the same problem with the FAA A&P exam. No solution except to learn the wrong answers.

EGHO-LFQF-KCLW, United Kingdom

So here is my experience of an ATPL question for performance taken in Feb at the CAA at Gatwick.

This is word for word how the question was written:

“You’re in a single engine piston aircraft:

Altitude: 9500ft
Terrain Elevation: 500ft
Gross Gradient: 11%
TAS: 250kt
Tailwind: 50kt

Still Air Distance = (Height Difference (ft)x100)/ Net Gradient (%)
Ground Distance = Still Air Distance x (GS/TAS)"

Answer box --NM

Now I presume that the question is to calculate net glide distance, but since they don’t ask a question how can any answer be correct or incorrect?

The exam was full of errors and half baked questions – I did a FOI request and was rebuffed with a number of points including detection of crime….

What have I learned about the CAA? Don’t bother questioning why, just learn the B/S that they want you to learn.

Learning & burning
Popham, United Kingdom

Unfortunately a lot of the body of knowledge in these civil service departments have been hollowed out. Personnel would have moved to EASA/EU departments pre Brexit, or retired, budget cuts, use of contractors, etc

Oxford (EGTK), United Kingdom

I’ve been told that “consultants” (contractors) are widely used for this.

I’ve met some over the years and if they have ever flown a plane, it was about 30-50 years ago.

Administrator
Shoreham EGKA, United Kingdom

Newbie wrote:

You’re in a single engine piston aircraft:
….
Altitude: 9500ft
TAS: 250kt

I am more interested where I can buy a (presumably turbocharged) SEP that cruises at 250kt at 10k / 210kt IAS, because it would do around 300kt at FL240… why bother with a turbine.

Or do they mean best glide speed? In that case, it would stall around 150kt or so, perhaps it is a Starfighter, which was single engine but a jet. Don’t think that would manage a glide gradient of 11%, though, it glides about as well as an anvil.

Biggin Hill

I had to calculate if the M&B was within limits without having the empty weight/arm info of the airplane…

jfw
Belgium: EBGB (Grimbergen, Brussels) - EBNM (Namur), Belgium

I had an altimetry question during my ATPL exams in which the answer one would select among the available options would depend on whether 27 or 30 ft was used as equivalent to 1 hPa. You could argue: “well then use the more accurate 27 ft, of course.” Sure but the same exam had other questions in which one would obtain one of the available options only if using 30 ft rather than 27.

I also used to keep a folder full of examples of terribly worded ATPL exam questions. Sometimes none of the available answer were correct, sometimes several or even all of them were…

EDDW, Germany

I have some examples of poor questions here and some recent examples from the UK CAAs scandalous “infringer punishment procedure” here.

When my A&P/IA was doing his EASA66 (using his A&P work experience as a credit, but still had to do the written exams) he showed me some of the questions in the electrical syllabus, and I was appalled how useless and wrong they were. They were written by someone who was literally clueless about electrics.

The ATPL QB (question bank) was generally confidential. The FTOs collected the bulk of it by assigning each student going into the exam room to memorise one question and then an FTO man with a clipboard stood outside the exam room and wrote them down before they were forgotten. So the big money-making FTOs produced their in-house copies of the QB. And nobody had any incentive to fix anything because each profited directly from their students getting better grades sitting the crap material than another FTOs students. It is same with open source software, which ranges from a buggy pile of crap to something “OK but needs a lot of fixes before it can be used commercially”, and the fixes almost never get contributed back because the business case is to not do that. Around 2007, two groups of students did a “Freedom of Information” legal challenge in Denmark and Belgium and got the real JAA QB, and later it was translated into English. This QB has since been “improved” but only in countries where somebody in the national CAA actually understood the subject (Germany and UK foremost I think). This QB has always been full of crap and almost the only way to pass it was by heavy computer study.

To finish it nicely, if you see some crap Q in an exam paper, you are allowed to report it and contest it (and get it credited) but you not allowed to record it so how the hell are you going to report it?? The system is stacked against you. In the CAA CAS bust exam (link above) it is possible to take a screenshot, because you are at home, but if you try to get it credited, the CAA busts chief (who is almost impossible to contact, and never replies to emails or letters) tells you to get stuffed, so again you get nowhere.

I don’t think anything has ever changed or ever will change in this business. The questions are prepared by people who don’t fly; they may be FTO lecturers (who also don’t fly). Reportedly, the JAA QB creation was, like the Eurofighter, distributed around the EU, and e.g. the Air Law was done by a Portugese ATCO! No “real pilot” will ever get anywhere near the flying QB and no “real engineer” will ever get anywhere near the maintenance QB. And as I say above the FTO / ground school business case is stacked against any improvement because the crap there is the more money these companies get. They need to skim 80-100k off each customer, and we all know the CPL/IR flight training cost is only about 30k-40k.

Administrator
Shoreham EGKA, United Kingdom
16 Posts
Sign in to add your message

Back to Top