Reading Inspire XML with FME

@SafeSoftware asked, if there’s any chance I’ll write this blog post also in English: http://gissiajapaikkatietoa.wordpress.com/2011/11/16/mmln-kuntarajat-fmella/. There’s a very good chance you can read it in English, since I’m now writing it in English. Finnish doesn’t seem to be the universal language yet, thus I understand @SafeSoftware’s question.

To begin with, according to Safe Software, FME 2012 can read XML documents in Inspire schemas out of the box, since FME 2012 contains Inspire schema files (from Annex I). This would mean that reading XML documents in Inspire schemas and writing to different formats is easier than before. This needs testing!

Downloading data

I’m testing the new functionality with open data from the National Land Survey of Finland.

I download municipality boundaries in XML from http://www.maanmittauslaitos.fi/ilmaisetaineistot.

Reading data

I choose INSPIRE GML as the format and municipality boundaries as the dataset with a scale of 1:100 000.

Inspire GML-luku
Next thing to do is to check parameters. I choose Dataset Only as the setting for Show Feature Types from:

As a result, I get two feature types, AdministrativeBoundary ja AdministrativeUnit. Complex properties are mapped now as nested attributes. For example following XML fragment

<inspireId>

<Identifier>

<localId>au13572990</localId>

<namespace>FI.NLS.AU100</namespace>

</Identifier>

</inspireId>

will form two attributes inspireId.Identifier.localId and inspireId.Identifier.namespace.

AdministravieBoundary ja AdministrativeUnit

Writing data

Let’s continue the test by writing administrative units and boundaries to different formats, e.g. MapInfo TAB, ESRI Shape, KML, PDF, DWG. I built the workspace and defined three parameters, with which I can direct the transformation of the data. The parameters are: Source file(s), Destination folder and Destination data format. The end-user doesn’t need to do anything else than define the values for the parameters and click OK.

Inspecting results

After running the workspace, we can inspect the results in FME Data Inspector or Viewer. For the first run, I chose MapInfo TAB as the destination data format, and the result has correct geometries and attributes:

Kuntarajat MapInfo TAB:ina.

Other formats that I tested were KML, PDF ja DWG, which worked without problems.

Reading XML data in Inspire schemas is now very handy with FME.

Leave a comment

Filed under Inspire

Real time bus visualization

Some of you might know that you can publish real time data to FME Server as a KML Network Link, and then visualize data in Google Earth. Safe Software (www.safe.com) has this really great real time taxi demo:

It generates taxi locations with a Python script, and FME Server is used to stream the locations and visualize them in Google Earth.

While I was fiddling with the demo, I throught, could this be used for visualizing buses, which get real-time passenger information, for example current number of passenger. So, I downloaded the demo Safe provides, and started modyfying it.

Modifications included creating 3D features from each bus location, and extruding each feature by number of passengers. Also, the number of passengers is visible on top of each feature with the bus number, for example, 111 | 30, where 111 is bus number, and 30 is the number of passengers.

The bus data is real time data, and is updated in every second. FME provides a really easy way to handle real time data, and publish it to applications such as Google Earth.

1 Comment

Filed under FME

FME 2011 launched!

I’m a week late with my blog post about FME 2011, but it doesn’t make FME 2011 shine any less. During the week after the release, I’ve been trying to understand, how Safe Software does it. FME 2011 is a very big release amongst earlier releases. It takes spatial data crunching to a whole new level. It’s easier, faster and has a lot new transformers and supported formats.

As I’ve found when reading blog or magazine posts about the launch, lots of them mention FME 2011 point cloud support to be one of the most important new enhancements. I agree. Lidar is now going to be mainstream or is already. This means very good times for FME. Read more here, http://www.directionsmag.com/articles/a-q-a-on-fme-2011/151615, how Dale Lutz describes the new point cloud support. He says point clouds are “misbehaving rasters” actually. He says adding raster support point helped them in adding point cloud support.

I’m also very happy for the Rest support for FME Server. Now it’s much easier to integrate FME Server to already existing web services.

Rest of the enhancements can be found here: http://www.safe.com/highlight/fme2011/whats-great/

Last but not least, FME 2011 World Tour starts from Vancouver and tours around the globe in 25+ cities. The events are free! That is indeed very generous. More information: http://www.safe.com/FME2011WorldTour

P.S. Now it’s time to start spamming Safe Software with new enhancement requirements.. 😉

2 Comments

Filed under FME

Smallworld Data Validation with FME

Hello, I built an FME process to validate couple of error types in Smallworld database. The errors that need fixing are:

  • Lines that don’t connect (I had some cable network in my database).

 

Cable lines that don't connect.

 

  • Intersections, that should have an object marking the intersection (In this case I had joints in the cable intersections, but some were missing).

 

A joint is missing from an intersection.

 

I read the cable and joint data to FME, validate it, find errors and report back to Smallworld. FME transformers that are really handy in inspecting geometry errors are

  • Snapper
  • LengthCalculator
  • Tester
  • Intersector
  • ListElementCounter
  • PointOnPointOverlayer

There are lots of transformers which could be used too, but I found those ones suitable for this validation case.

Here’s what the result looks like in Smallworld:

 

The red flag indicates cables that don't connect, and the orange flag indicates a missing joint.

 

This process is quite handy to validate invalid geometries and missing features. Data is read to FME from Smallworld, validated, and written back to Smallworld with just one click. After that the user fixes the features. Of course FME could also do the fixing, but here FME is working as a validation and reporting tool.

Leave a comment

Filed under Smallworld

From Smallworld to 3D PDF

I’ve worked with Smallworld for several years now. One of the useful ways to visualize all kinds of spatial data is utilizing 3D PDF. PDF itself is widely known format, thus why not use 3D PDF for visualizing data derived from Smallworld database. First I built some electricity network in Smallworld (Poles and cables), then forests around them and a few buildings. Poles, forests and buildings had height attributes, so that helped to transform them from 2D to 3D. Also I had a DEM from the area, so the task was to transform all the features to 3D and place them on the DEM. Also, a raster was added on the DEM.

The main challenge

The main challenge was how to transform data from 2D to 3D world. FME has a couple of useful transformers for constructing and transforming 3D objects: Extruder and SurfaceDraper.

Extruder stretches your 2D features to 3D by a specified height or a vector.

SurfaceDraper places your 2D features on an elevation model. You should use SurfaceDraper first and then after that Extruder.

An other challenge were to make the cables hang. It worked well with a bunch of transformers. The result of the hanging cables can be seen in the picture.

Here is a picture of the result, where you can see the cable, forests and buildings in the left.

Here is another picture of the result 3D PDF.

From Smallworld to 3D PDF with FME

1 Comment

Filed under 3D

FME Scandinavian User Conference 2010

Over 110 FME enthusiasts, lots of presentations, great conversations with FME users and Safe guys, FME 2011, good food and of course So you think you can FME. That’s what happened in Malmö, in FME SUC 2010. Thank you Sweco, thank you Safe, and other organizations and participants!

Ken and Dale did a remarkable job with presenting FME Server, CAD <-> GIS use case, and FME 2011. Ken and Dale gave separate presentations, when previously Don&Dale have given the main presentations together. I guess some hyperventilation was reduced in this way. Ken pointed out some top challenges people face when moving between CAD and GIS worlds. Styles, valid geometry, and valid attributes. It was also interesting to hear that when Safe went to this one GIS fair, they had CAD and GIS words on their posters for the first time, people were stopping by their booth a lot more than previously. CAD to GIS is still one of the main largest needs for data transformation. And FME does CAD and GIS transformation well!

FME 2011 is going to be awesome! http://www.fmepedia.com/index.php/Whats_Great I have already used the beta, but I didn’t really know all the new stuff, that Safe’s been working on. FME 2011 has inspection point functionality. You can set inspection points to any link, and really see what features go to the inspection point. It’s really handy to use for debugging. It has a window, where you will see the feature, and below it has the feature attributes. You can even set rules. For example, if you know that certain type of features are sensitive for crashing, you can make the inspector stop to first one of those type of features, and you can check, why it crashes. Clever!

I’ll try to post more of the conference later, when I get to see the presentations again. I lost my notebook there, and I need to through the presentations again, and hopefully I then recall the visions and ideas that I got in the conference. Stay tuned.

3 Comments

Filed under FME

Finnish FME User Day 2010

We had our first FME User Day yesterday, and I realized, that Finnish FME users are almost getting ahead of me in FME knowledge. It’s frightening but at the same time challenging. Many users were very intrigued by XMLTemplater and all the other new XML / GML functionality in FME. I guess one of the top problems is, how to obtain complex XML from various data sources, when the source schema and the XML schema change. XMLTemplater and SchemaMapper really make handling XML and GML much easier.

It’s not no to XML anymore!

Some were also interested in reading WFS, and filtering the WFS data in FME. All in all, there were a lot of discussion, and the users are looking forward for the next Finnish FME User Day 2011!

Thank you very much Safe Software and Dean for your commitment to the user day!

Leave a comment

Filed under FME