Saturday 3 April 2010

Draft Proposal

Better Print Support

Personal Information

Name: Carlos Enrique López Garcés.

Country: Mexico.

School and degree: I study at the Instituto Tecnológico y de Estudios Superiores de Monterrey in Guadalajara, Jalisco. I am currently studying the eighth semester of a bachelor’s degree in Computer Technology Engineering. This degree program is focused mainly on the areas of Computer Graphics and Artificial Intelligence, therefore many of the courses are based on these topics. I expect to graduate in December 2010.

Email: **************.
Phone: ##############.

Project Details

Title: Better Print Support.

Introduction:

'Better Print Support' is a project idea aimed at solving the current limitations that users find in Mapnik when printing maps to PDF documents. These limitations are minor, since there is already a strong development inside Mapnik's core for addressing these issues, but important to be considerer for achieving optimal printing quality.

Background:

There are currently two desirable features that members of the Mapnik and related GIS communities have asked for and that would be suitable for implementing over the period of time set aside for Google's Summer of Code: Finer control over scaling and resolution and better post-processable output. The motivation for choosing such features are presented next:

Finer control over scaling and resolution

According to the description of the project, many applications that use Mapnik to offer their services are also interested in obtaining support for printing directly from it. One such application is TownGuide [1]. This client application relies on Mapnik for rendering a high quality map in a digital image format, such as PNG and JPEG, which is later processed by a third-party PDF generator to produce PDF output.

The problem encountered by the author of this application was that adjusting the resolution at which the map was rendered caused the image to have a loss in quality, evidenced by 'pixelated' road casing lines and blurry edges. The adjustments of resolution were performed in a rather ad hoc manner: To achieve a higher resolution, directives in Mapnik were configured to produce the map in a larger size and resized later to fit a smaller area in the PDF document. This method had the counterproductive effect of displaying the text of streets and cities in small font sizes, since they were rendered by Mapnik according to the size that was originally requested.

Illustrations 1 and 2 show a portion of the same map at different resolutions. As it can be perceived, a map printed at a lower resolution following the previously outlined method exhibits thick road casings with pixelated borders and blurry dotted coast lines. On the other hand, a map with higher resolution displays the map with elements that do not appear in the lower-resolution one.



From [2].


From [3].

This problem arises from the fact that resolution is being manipulated outside Mapnik's scope, using external libraries for modifying an image that is not scalable.

Better post-processable output

Vector-format images are more flexible than raster-based ones when it comes to editing them after they have been rendered. A vector image in a common format can be processed by photo editing applications to meet the special needs of users. In this case, a map in a vector format could be zoomed in and resized without loss of definition in the image. A user could also be able to draw over the map to add important visualization aids and cartographic elements (grids, coordinates, scale bars and suchlike) in an easy and flexible way. 

Applications based upon Mapnik that do this kind of processing are TownGuide and MapOSMatic [4], which also produce output in PDF format. Since PDF is also vector-based, the integration of maps in vector format, like SVG, would be seamless and smooth.


The idea:

I have divided this section, as above, into two subsections, each addressing one of the features intended to be developed in this project.

Finer control over scaling 

There are currently two main renderers in Mapnik's core, one based upon the Antigrain Geometry graphics (AGG) library and the other upon Cairo. The AGG renderer has only support for output in raster format (JPEG and PNG), whereas the Cairo renderer exploits the concept of surface [5] (the target on which drawing operations are performed) found in Cairo to define an abstract interface behind which different image/document formats can be implemented for rendering, both raster and vector (PDF, PNG, PostScript, SVG, etc.) 

With respect to the raster or bitmap formats, it is widely known that they are not easily scalable, so the challenge is to investigate, design and implement an algorithm for scaling raster. Using a library for such purpose is also an alternative that must be evaluated.

The implementation of such a scaling mechanism is needed when it comes to adjusting the resolution at which the image of a map is required. The goal is to make possible to render the maps at high resolution, so the quality of the image in the PDF document is comparable to that present in other formats. Once the scaling of raster images is implemented, a client application would specify the desired resolution, which would then be expressed as a scaling factor for determining the size of symbols. This has already been tested for ticket #343 and the idea would be to determine the way resolution parameters will be exposed to the user.

Another issue associated is that of variable units. Measurements and quantities could be easier to handle if they were expressed in units other than pixels. This would require the implementation of functions for converting between each of the new units to pixels. Once the conversion is done, everything would be handled in pixels internally. After the internal processing is done, a last conversion would be applied to display everything that is measurable in the map in the units specified by the user.

The project will then consider:

  1. Further research in Mapnik's code base and trac to determine the extent to which scaling has been implemented. Tickets #259 and #343 address a portion of the work that must be done in this area.
  2. Addition of an interface that would be exposed to the user for specifying resolution. This interface would be generic enough to be used across different renderers and would be reflected as a parameter added to functions and stylesheets. This parameter would be used internally to scale symbolizers, line strokes and fonts. With respect to text scaling, research inside the project would determine the best solution for the problem outlined in the 'Background' section of this document.
  3. Implementation of an algorithm for scaling raster symbols if needed (this point needs to be investigated further to determine its applicability in the project).
  4. Addition of an interface for specifying units, both in input and output. This point includes the implementation of a set of functions for converting from user units to pixels. The units considered are cm, mm and in (see ticket #389). 

Better post-processable output

A third renderer would be implemented to produce output in SVG format, following the model of agg_renderer and cairo_renderer, which are subclasses of feature_style_processor. This will require the implementation of the methods for processing and rendering each basic symbolizer. For producing the output, Boost::Spirit::Karma would be used to increase performance.

The project will also consider:

  1. Learning the Boost::Spirit::Karma library.
  2. Design and implementation of an SVG renderer, based on Boost::Spirit::Karma. The use of Karma as a basis follows the need of producing SVG output efficiently [6], since cairo_renderer already produces SVG output, but does so in a rather slow way [7]. 
Goals:

As it can be observed, the project considers several important goals, which can be categorized as belonging to one of the following ends:
  • Finer control over scaling and resolution.
  • Better post-processable output.
In order to make this project achievable, I've decided to prioritize the implementation of the ideas in the order suggested above, that is, the main goal of the project is to offer client applications an interface for specifying the output resolution. The reasons that drive this decision are:
  • There is a project for OpenStreetMap, called 'Easy Printable Maps', that may rely upon the improvements in resolution flexibility [8]. One of the key features provided by this project will be the possibility of printing maps to different paper sizes. In order for the quality of a map printed in large format to be preserved, high resolution images must be produced. This projects may require support for variable resolution pretty early in its development.
  • Many client applications that use Mapnik as backend [1] and that don't make use of its vector features would benefit without considerable changes to their code bases.
The above decision also follows my concern for making this project feasible within the time frame given to it.

Project plan (how do you plan to spend your summer?) :

Future ideas / How can your idea be expanded?:


Explain how your SoC task would benefit the Mapnik community and what other projects might benefit directly or by example:


The possibility to gain control over the resolution at which a map is rendered will certainly increase the quality of the output image when printed to PDF documents. Furthermore, some of the issues that the project would cover have been asked for attention by members of the community, a sign of the interest in the project.

With respect to other projects, I have already mentioned the relationship between this project and 'Easy Printable Maps' of OpenStreetMap. TownGuide would also benefit from higher-resolution maps, which may improve the quality of the printed maps this application produces.


Experience

Please provide details of previous experience with C++, python:

C++

I learned the foundations of the language through a school project/program for the 'Operating Systems' class developed in the C language. The project had the goal of making use of the Windows and POSIX APIs for making different system calls (to work with threads, files, interprocess communication methods, etc.) Although I did not have the chance to work with the object-oriented concepts introduced by C++, I acquired the basic knowledge of pointer/reference handling and memory management.

Formally, I started using the language in the 'Algorithms' class of the degree program, were I implemented algorithms for sorting sequences, processing text, and analyzing graphs. Since 'algorithms' is the topic in computer science that I am most interested in, I have dedicated much of my spare time to implement other classic algorithms (with C++ as my preferred programming language for doing so).

Finally, the internship I am doing at Continental Automotive this semester has allowed me to practice and learn the language. The project I am working on is about an application for analyzing the results of various kinds of tests performed on hardware.

Python

I learned the language during the 'Computer Graphics' course of my degree program, in which I used PyOpenGL, an OpenGL python binding, to program a simple 3D editor and a flight simulator. In addition to PyOpenGL, Numpy was also introduced to me during this class.

After that course, I began to use Python for projects of subsequent courses: For the 'Programming Languages' class, I chose it to implement a simple video game using PyGame, a Python library for game development. For the 'Computational Intelligence' class, I used it to program some algorithms of the soft computing area, including 'Ant Systems', 'Genetic Algorithms' and 'Simulated Annealing'.

Please provide details of previous experience with the Antigrain Geometry Library, Cairo, or Boost:

Unfortunately, I have no experience in programming with the Antigrain Geometry library or with Cairo, though I have been studying the latter during the past few weeks.

Boost

The application I am working on in my internship this semester is strongly based on the following libraries:

  • Boost::Exception
  • Boost::Foreach
  • Boost::Lexical Cast
  • Boost::Optional
  • Boost::Property Tree
  • Boost::Smart Ptr
  • Boost::String Algo

Please provide details of previous experience with mapping or GIS:

I have little previous experience with mapping and GIS. My knowledge of mapping and cartography is comprised of what I have learned since I first heard about Mapnik in the Google Summer of Code program website and of what I have read in a book called 'Web Mapping: Using Open Source GIS Toolkits' by Tyler Mitchell (available in Google Books).

I must admit that my knowledge in this area is limited, yet I am not completely unfamiliar with Geographic Information Systems: Two years ago, I participated in the development of a web application that processed GPS information from a server that was fed in real time with data sent from a GPS device connected to the computer of a vehicle. The application was responsible of displaying the points in a map of a city that were visited by the vehicle in order to trace routes and calculate travel times and distances. The maps were provided by Google and accessed through the Google Maps API.

Please tell us why you are interested in GIS and open source software:

GIS

GIS attracted my attention because cartography is a discipline founded on mathematics and geometry that actually applies the knowledge, theory and techniques of these sciences for solving a practical problem. The translation of a mathematical problem into a solution modeled by a computer program is also something I have always been very enthusiastic about. Due to the nature of my degree program (technology-oriented and with a strong focus on research), the opportunities to do so in a school project have been few, though. Hence, participating in this project would give me the opportunity to explore such approach further.

Another reason for choosing GIS, besides its mathematical basis, is the fact that much of the emphasis of my degree program is on computer graphics and GIS applications are much about dealing with image manipulation and analysis. Participating in a project of this kind would allow me to exercise and increase the knowledge I have of this area, and interacting with people experienced in the field would certainly benefit my learning.

In reference to Mapnik, my interest in it grew stronger when I saw the quality that could be achieved in the image output and the widespread renown it has among web-based applications offering mapping services.

Open Source


The primary reason I am interested in open source software is that projects of this kind are unlikely to be found when working for an enterprise.

Please tell us why you are interested in your specific coding project:

I consider that this project is just as important as any of the other projects presented in Mapnik's list of ideas, since they are all directed towards increasing the range of functionalities and services offered by this software, and improving the quality and flexibility of the code. However, 'Better Print Support' has a direct impact on other open source projects like OpenStreetMap [], Walking-Papers [], and MapOSMatic, whose developers are looking for a way to easily adapt the output to various paper sizes without loss of quality.

This project was suggested to me by Waldemar Quevedo, a student who is applying for a project in OpenStreetMap. The project he is applying for, titled 'Easy Printable Maps', is a potential first user of the features related to resolution. I consider that this relationship between projects is beneficial because some design issues associated to this project need to be defined in a way that is both flexible to be reused across components inside Mapnik and intuitive for the developer that uses Mapnik as a basis for implementing another service. The latter would be a condition that would be continuously tested and reviewed in order to ensure proper application in the other project (Easy Printable Maps).

Do you understand this is a serious commitment, equivalent to a full-time paid summer internship or summer job?

I understand the commitment I make when presenting this proposal. I am also aware that mentors kindly share their time for making this project possible. Therefore, I commit myself to following the rules set by the organization. My intention is to contribute to the Mapnik's project in the best possible way.

References

[1] TownGuide: http://www.townguide.webhop.net/

[2] TownGuide - 100 dpi map: http://www.townguide.webhop.net/output/73/townguide_poster.pdf

[3] TownGuide – 300 dpi map: http://townguide.webhop.net/output/72/townguide_poster.pdf

[4] MapOSMatic: http://maposmatic.org/about/

[5] Cairo Surfaces: http://cairographics.org/manual/cairo-surfaces.html

[6] Karma's Performance: http://boost-spirit.com/home/2010/03/09/integer-to-string-conversion-karma-fastest-again/

[7] Better Print Support discussion at mapnik-devel mailing list: http://www.mail-archive.com/mapnik-devel@lists.berlios.de/msg00645.html

[8] Easy Printable Maps: http://wiki.openstreetmap.org/wiki/GSoC_Project_Ideas_2010#Paper_Output_Projects

http://www.pygame.org/news.html
http://numpy.scipy.org/
http://pyopengl.sourceforge.net/
http://www.amazon.com/Desktop-GIS-Mapping-Planet-Source/dp/1934356069  
http://old.nabble.com/Re:--OSM-dev--Mapnik-output-resolution-td27495133.html
http://www.mail-archive.com/mapnik-devel@lists.berlios.de/msg00646.html
http://www.mail-archive.com/mapnik-devel@lists.berlios.de/msg00649.html
http://www.mail-archive.com/mapnik-devel@lists.berlios.de/msg00140.html

2 comments:

  1. Great to see your proposal coming together!

    Here are my comments:

    * First Paragraph: the problem is not just when printing maps to PDF. Certainly many people target PDF output when going to print but all formats require advances to support high resolution output/scaling. But, I understand why you stated it this way, because the example of townguide pushing PNG output into a PDF certainly highlights easy problems that needs fixing. The more advanced way to provide PDF output is to render all map features as vectors into the PDF (or SVG). Even thought the PDF format can support embedded rasters, usually when high resolution output is desired the PDF will look better and be of smaller size when all vectors are used.

    * I like the way you've divided up the application into "control over scaling" and "post-processing"! Very nice.

    * "Better post-processable output" section says: "photo editing applications". Really people are using vector editing or vector drawing applications like inkscape and illustrator, not photo applications (which work on rasters). I'm sure you understand this, its likely just a typo...

    * You say "so the challenge is to investigate, design and implement an algorithm for scaling raster". Actually I think it is easy to scale rasters, and in fact the algorithms to do so (always uses to scale down) are already available inside mapnik (http://svn.mapnik.org/trunk/include/mapnik/graphics.hpp_). The trick is that certain objects need to be scaled BEFORE being painted onto raster surfaces. In some cases this is easy, e.g. if higher resolution output is requested (to be rendered by AGG) then larger fonts and line widths can be fetched before rendering, etc... I've not given it much thought, but potentially the "scaling" factor will need to be different depending on whether the output is raster (via AGG) or vector (via Cairo). Not something that needs to be solved now of course, but a fun challenge to test this summer :) Symbols are different of course, because currently mapnik only supports raster symbols which, if scaled up, look bad. But you cover this later on, so let's not worry about this to much now...

    * The next several paragraphs in that sections are great!

    * Re: "Implementation of an algorithm for scaling raster symbols if needed (this point needs to be investigated further to determine its applicability in the project" - this is good enough for the application. We can talk more about this. In particular Tom Carden has some ideas that might help that he is researching. And Artem has recommended using AGG SVG parser to read symbols in a format that can then be easily scaled.

    * in relation to writing a new svg_renderer: I talked with Artem more about this today and I want to confirm that I think this is a very good idea that it should be feasible to have a part of the project, with Artem's guidance, so he will be the primary contact in advising this. However, you say "cairo_renderer already produces SVG output, but does so in a rather slow way [7]. " I think a native svg authoring renderer certainly could be written to be faster than cairo, but the main idea is that we need something more flexible that Cairo can provide, to be able to customize our output for more of the varied usecases of SVG/Post-processing. We can talk more about these later.

    * I really admire and support your ideas about how your skills can contribute to Mapnik and GIS software as a whole. Bringing Math and Graphics skills to GIS problems is exactly the basic of this project, so great stuff.

    * I also think the collaboration with Waldemar is fantastic - thanks for making this possible.

    ReplyDelete
  2. Call us at we24support Lexmark Printers Support number at 1-866-978-0799,Enhance your system
    functioning and keep it clean with we24support services.
    Lexmark Printers Support
    lexmark printer technical support

    ReplyDelete