Menu Sign In Contact FAQ
Banner
Welcome to our forums

GRAMET (merged thread)

It works for me…

At this time we require registration as requesting a GRAMET consumes quite a bit of computing power and we have to protect the infrastructure. Now if you provide us with an SSE assembler optimized JPEG2000 tile decoder that would allow us to speedup the GFS processing by factor 10 or so, we might rethink that position

Router’s GRAMET today at route LDZA→LDLO and back was 100% correct – it’s not usual that on such day, with lot of convective activity, forecast is so precise

LDZA LDVA, Croatia

Now if you provide us with an SSE assembler optimized JPEG2000 tile decoder that would allow us to speedup the GFS processing by factor 10 or so, we might rethink that position

You shouldn’t need to do it in asm. Doing it in C and using a complier with a decent optimiser (the Intel compiler has a pretty good reputation, I’ve not used it though — if I’m writing C I usually also care about portability so I tend to avoid CPU specific or proprietary compilers). The compiler will almost out optimise an asm programmer these days, and you’re certainly never going to get an order of magnitude improvement hand coding asm today. I’d be surprised if you can beat libjpeg by much even then, so it’s possibly not even worth rolling your own.

Anyway, back to the GRAMET. I’ve had a quick look at the site – is there any way to generate one without entering an IFR routing (IFR routings are kind of pointless for me). I’d be interested in a general view either for a single location or a straight line between two points.

Andreas IOM

Was perfectly accurate today (02/05) for me from North UK to North France. Was vertically accurate to +-1000’ and laterally accurate to +- 20nm

United Kingdom

Doing it in C and using a complier with a decent optimiser (the Intel compiler has a pretty good reputation, I’ve not used it though — if I’m writing C I usually also care about portability so I tend to avoid CPU specific or proprietary compilers). The compiler will almost out optimise an asm programmer these days, and you’re certainly never going to get an order of magnitude improvement hand coding asm today. I’d be surprised if you can beat libjpeg by much even then, so it’s possibly not even worth rolling your own.

That is generally not wrong but not in this case. JPEG2000 and similar algorithms can greatly benefit from SIMD (single instruction multiple data) and compilers can not use this technique without manual instructions. Some of them are able to use it for floating point arithmetics and to vectorize very simple loops but that’s not what would buy us much here. The standard JPEG libraries like libjpeg-turbo greatly benefit from SSE but I haven’t found a library doing JPEG2000 which is used by GFS (and almost nobody else). In a past life I have worked on optimizing JPEG encoding down to the last CPU cycle…

Anyway, back to the GRAMET. I’ve had a quick look at the site – is there any way to generate one without entering an IFR routing (IFR routings are kind of pointless for me). I’d be interested in a general view either for a single location or a straight line between two points.

Yes, semi official interface for now: http://router.euroga.org/gramet

Last Edited by achimha at 03 May 06:55

Could the colours be changed slightly so that it prints on a monochrome laser? Currently, some features are totally invisible.

I used to have a duplexing HP colour laser but the refills were £250, the fake Ebay ones were £90 and clogged it up with muck, and eventually it stopped working totally. A real POS, and it wasn’t a totally cheap one – about £500. Now I use a duplexing inkjet but the large areas of colour waste ink so I print most things in mono.

I agree about assembler – it’s not so much that C generates a lot more code but that in assembler one can use various tricks and side effects.

Last Edited by Peter at 03 May 07:04
Administrator
Shoreham EGKA, United Kingdom

It’s not about tricks or side effects in this case, it’s about whole functional units of modern CPUs, vector units where one instruction can manipulate a whole set of data. Traditional compilers can only use these functions in a very limited way because the high level language cannot express the whole vector business and it doesn’t have the intrinsic data type of the vector unit.

It’s a lot of work but I hope somebody will do it for JPEG2000. This format was supposed to become a standard, replacing the traditional JPEG fixed block size discrete cosine transformation with the more modern wavelet compression but unfortunately the format never took off, albeit being an ISO standard. To calculate the GRAMET image, we spend around 80% of the time doing JPEG2000 decompression.

Here’s a good SIMD/SSE implementation of classic JPEG: http://sourceforge.net/p/libjpeg-turbo/code/HEAD/tree/trunk/simd/

Last Edited by achimha at 03 May 08:55

Now if you provide us with an SSE assembler optimized JPEG2000 tile decoder that would allow us to speedup the GFS processing by factor 10 or so, we might rethink that position

Wait a cotton pickin’ darn minute… decoder? Which means to say you’re having to get the data for this effectively by decoding and OCRing a JPEG image, not from a raw data feed where the data is in some intrinsically machine processable form?

Andreas IOM

Well, note quite, GRIB2 is the standard by the WMO (World Meteorological Organization) and contains bitmap data but in the mathematical not “user viewable image” sense. This data can be encoded by different CODECs, one being PNG and the other JPEG2000. The US met office NOAA use JPEG2000 unfortunately. I guess when they did this in the 1990s, that “2000” sounded very modern

It’s not too bad, we can do a GRAMET in a few seconds from the raw GFS data. Ogimet translates and pre-computes the data which is why it is faster but also why it is offline for several hours a day. We could do the JPEG2000 decompression once every 6 hours for the whole dataset but I don’t know how many hundred GB that would require and how long it would take. As long as our GRAMET takes less time than logging in to some other routing site, we’re probably fine However, this is why you need to be registered and logged on to access the GRAMET.

Last Edited by achimha at 03 May 15:46

Wait a cotton pickin’ darn minute… decoder?

Given that weather forecast data is obviously raster data (values on a two dimensional grid), it’s not exactly far fetched to use an image compressor to compress forecast data. That’s what the WMO did.

LSZK, Switzerland
Sign in to add your message

Back to Top