About mid way through the year we had to write an essay on a research subject, I chose texturing and this is the video I made to go along with the essay:
The essay is below the cut, it is the first I have written since High School and I am pretty proud of it, a lot of the information I couldn't fit into the video is there.
Abstract
For my research project I aim to research the question, 'What are the best current practices for the creation of asset textures in 3D film?' I am hoping to look at the current workflows, practices and programs used today to develop textures for assets, along with the past methods and history of the art to further explain and learn how current practices came to be.
I believe this topic is important for me to focus on this year, as while I practice modelling often, as this is what I wish to specialize in, I rarely texture, and often finish a model and leave it un-textured. I find myself avoiding texturing as I have troubles with it and I believe this is because I don't yet understand how to properly create them. Despite this, I would love to be able to texture to finish pieces off nicely for my portfolio, and I think it is important for me to know how to do it, as well as the current workflows and programs I should be looking into using.
In regards to methods I wish to use, I plan to research heavily online, while developing my understanding by applying this to my own work as I go. I want to look into the past texturing of feature films by studios such as Pixar and Weta Digital, through to their more recent films and texture practices. I will look into forums, as well as tutorials and past 3D World magazine issues, to get an idea of alternative programs to use besides Maya and Photoshop, along with workflows that are used such as the use of projection textures, which we have not learnt in class. I will also ask questions to experts or people I find inspirational or have higher knowledge than me, in hope they can recommend tutorials, and to get their own personal opinion on the texturing practices of today and how its advanced from the past methods. Furthermore, I will look towards the future of texturing and what I believe is next for the art. Throughout this I will apply my knowledge to my own assets to help cement the information the ultimately come to a conclusion about the best current practices.
The Past
The texturing of environmental assets within the computer generated and visual effects industry has always been considered one of the most important elements. It is something that drives the story and allows objects to integrate with their surroundings both fluidly and seamlessly. It is a major part of any 3D film pipeline, and is a field that allows for increased artist expression and creativity as the industry and programs available expand. However, texturing in the past was little more than an afterthought that usually consisted as block colours or computer generated ‘shaders’ until more advance methods were later introduced.
Texture mapping was first pioneered by Edwin Catmull in 1974, as was bump mapping a few years later in 1978 by James Blinn. It wasn’t till 1984 however, that an asset, or in this case a spaceship, was shown ‘textured’ in the film ‘The Last Starfighter’. Despite this, due to the technological restrictions at the time the ships surfaces were limited to phong shading only, without any actual texture maps. This shader, named after it’s developer Bui Tuong Phong, allowed shiny reflective surfaces to be smooth. This technique of using the 3D softwares inbuilt shaders to texture assets then continued, despite the research papers Catmull had put into the art. Texturing however was finally taken to a new level when in 1986 Lucasfilm’s 3D Graphics department was purchased by Steve Jobs, who made Catmull president of the company now know as Pixar. Later that year they produced ‘Luxo Jr.’ a short film which was designed to advertise the ‘Pixar Image Computer’, a product they were selling as they were a software company at the time. In this short, the lighting and colours, which were applied through shaders, were calculated with Pixar’s own program, ‘RenderMan’. Later in 1987 however, an improvement was made, and ‘RenderMan’ now allowed computer artists to add colour and textures to 3D assets, as well as produce realistic photo images with it. This was a huge step forward for the history of visual effects as a whole, not just for texturing, and with Pixar’s first example of what ‘RenderMan’ was capable of in their 1988 short, ‘Tin Toy’, they won the Academy Award for Best Short Film. The short used photo textures, such as wood grain on floor boards, and even actual photographs in picture frames and magazines, a big change from the usual shaders audiences were use to.
Textures were taken to even further levels on characters in early 90s films such as ‘Terminator 2’ (1991) and ‘Jurassic Park’ (1993), but environmental textures were not again updated until 1995 with the first feature length animated film, ‘Toy Story’. A new high level of detail was achieved in this film, with elements as simple as the asphalt on the street being put together by Pixar scientist Loren Carpenter who “merged several photographs of gravel and sand in such a way that makes the texture look random enough to be real”.
By 1997 the texturing of 3D objects was becoming well known, and as explained in Rosalee Wolfe’s SIGGRAPH paper ‘Teaching Texture Mapping Visually’, the current processes regarding it was through the use of UV mapping and then using procedural textures or images to apply any artistic manipulation to the assets. Catmull’s 1974 development of UV mapping, the process of using the u- and v- parameters of the surface to normalise device coordinates, was becoming widely regarded as a way to stop distortion from occurring on the textures of surfaces. Meanwhile, procedural methods, that was from inside programs such as ‘RenderMan’, were being used for both texture and bump mapping, and outside images were inputted at the extremely low value of 512x324 pixels. After ‘Toy Story’, Pixar, as well as other animation studios such as DreamWorks, continued using a combination of procedural textures and photo-realistic images in their productions, most noticeably in their 1998 film ’A Bug’s Life’. However, hand-painted textures eventually increased in popularity, particularly in the early 2000s, with Pixar using this extensively on their 2004 film, ‘The Incredibles’. Even in different studios with alternate pipelines and programs, a similar approach was taken to ultimately give the film it’s required textures. In the 2004 motion-capture animated feature, ‘The Polar Express’, hand painted and photorealistic textures were used in combination with software developers MAXON’s ‘BodyPaint 3D’, which allowed texture artists to UV map and then paint textures directly onto assets within the one program, rather than using two programs, such as Autodesk’s ‘Maya’ for 3D work and Adobe’s ‘Photoshop’ for painting and photo manipulation.
Meanwhile, for the more visual effects based movies, the use of shaders combined with photos to ensure models looked photorealistic continued to be used. In the 2003 film ‘The Matrix Revolutions’ a battle sequence featuring 100 modelled ‘mech’ robots took around five-weeks to render, with each one having around 2000 mirror surfaces on it and 12 gigabytes of texture maps. Furthermore, on Peter Jackson's 2005 'King Kong' around 400,000 textures were made for the New York City set Weta Digital, the team who also worked on the Lord of the Rings trilogy, constructed from old photos and aerial surveys of 1930s New York. Two different softwares were designed by Weta's team for this film, one by Michael Baltazar which estimated buildings material types by analysing the luminance values in black and white photos from the time, and another by Jean Matthews which baked buildings into textures, allowing “buildings [to be] rendered with textures rather than 3D geometry” according to 3D CG Supervisor Chris White.
After 2005, CGI and VFX films were no longer a novelty, and now an integrated part of our film industry. 2007 saw the release of blockbusters such as 'Transformers' and 'I Am Legend'. Pixar's 'Ratatouille' also came out, which is now commonly referred to as one of the most aesthetically pleasing looking CG films, particularly due to their use of highly detailed hand painted, photorealistic and procedural textures. ‘Ratatouille’ also introduced the use of Ambient Occlusion to Pixar’s rendering process, which is a form of ray-tracing to create darkness and shadows in corners and the intersections of mesh. Belinda Van Valkenburg, the Shading Art Director, expressed that “Paris is an old city… there are so many details to an aged look… we had to put lots and lots of layers on top of each other to make sure that they all came together in a nice graphic way”. Even 'Transformers' relied heavily on their texture artists to add tyre treads, dirt, scratches, colour and other textures to the mechanical robots, and in all 34,215 texture maps were created for the production.
The Present
In 2009 James Cameron's 'Avatar' was released onto the world, a box office record-breaking film with a staggering 2500 VFX shots. A project led by Weta Digital, it required 900 workers, with 54 artists to create 2.5 million textures. Before beginning production however, Weta were tasked with finding a texture painting program that would be capable of dealing with the level of detail Cameron was after for his film. Jack Greasley, the senior R&D software engineer, recalled that they did “an evaluation of all the available painting softwares”, but without finding one capable enough, they eventually created the now widely used software, 'Mari'. ‘Mari’ is a 3D paint tool that allows artists to paint directly onto 3D models, called texture projection, without the need to flatten them to 2D paintings with the use of UV mapping, taking over ‘Photoshops’ usual role in the pipeline. Furthermore, Weta used 'Maya' as their 3D program, the then and now industry standard, and ‘RenderMan’ to render. Overall, almost every asset within 'Avatar' was painted in ‘Mari’, and texture sets could be several tens of gigabytes with hundreds of 4k pixel maps applied at once, with the biggest asset having 30gb per map. Lead Texture Artist Micheal Cox expressed that ‘Mari’ allowed artists to “paint a level of detail never before attainable in other paint packages”.
In addition to this new era of texture painting brought on by Weta's 'Mari', projection painting was also used in the 2009 film 'District 9' using the program ‘BodyPaint 3D’. While not used exclusively, it was still a major part of their film pipeline. Image Engine, the visual effects studio who worked on the film, used a combination of ‘Photoshop’ and ‘BodyPaint 3D’ to hand paint the aliens textures, with around 20 different variations of decals created to put interchangeably onto the creatures.
A year before 'Avatar's release, Walt Disney Animation Studios had also been working on a projection texturing software, with the name of ‘Ptex’. They first announced the software by demonstrating it on a T-Rex, before then putting it to use in their 2008 film, 'Bolt'. Rather than being pushed by the need for more detail however, like Weta created ‘Mari’ for, Disney created ‘Ptex’ as a way to fasten the pace of the pipeline, taking out the middle-man between modelling and texturing. This was UV mapping, which sometimes can take up to hours of work, all the while including seams and possible distortion. But ‘Ptex’ completely eliminated the need for any of that. Principal Software Engineer of Disney Brent Burley also explains that they were “tiling [their] models and putting multiple UV tiles on each surface… [it was] killing our servers with Render IO”, so much so that Disney found themselves avoiding image based textures and relying on procedural texturing before ‘Ptex’ was designed. ‘Ptex’ allows for texture maps to have different sizes, so the colour map can have a different resolution to the bump map, despite being on the same asset. Because of this, textures at Disney are tiled so they can go to 8K to 16K, going up to dozens of gigabytes. ‘Ptex’ is now the primary texture mapping method for Disney and has been used on all their feature films since 2008, and is also now used at Pixar as well. In 2009 it was made part of their ‘RenderMan’ software, and later in 2010 it was released online as a free open source product.
Despite the technological advances and hype surrounding projection texturing however, programs such as ‘Mari’, ‘BodyPaint 3D’, and ‘Ptex’ have not yet become industry standards, and UV mapping and ‘Photoshop’ continues to make up the majority of the texturing pipeline. This is as UVs are flexible, working in every 3D application out there, whilst ‘Ptex’ is only compatible with Maya, and UVs can be transferred on models. Taking the place of projection mapping, many other programs that also aim to make the process of UV mapping quicker exist, such as 'UV Master' and 'UV-Layout'. As such, all current and non-program specific environmental texturing tutorials, such as Ryan Ribot’s ‘Hand Painted Environment Texturing’, Leigh Van Der Byl’s ‘Environmental Texturing 3D Workshop’, and Peter Sekula’s ‘Advanced Environment Texturing Methods’, continue to use Adobe’s ‘Photoshop’ and UV mapping for texturing, with some mentioning that ‘Mari’ can be taught only personally requested.
Furthermore, UDIM mapping, a type of UV mapping invented by Weta in 2002 that ‘Mari’ uses, is becoming more popular, even overtaking programs such as ‘Ptex’ amongst studios large and small these days. UDIM-UVs is a way of creating a single linear number that identifies each integer block in the UV space, allowing for multiple UV tiles to be easily exported between programs. Big studios such as MPC, Double Negative, and Weta use UDIM daily, and it doesn’t look like it will be replaced by Ptex anytime soon.
These days, the main differences between asset texturing for VFX based features, versus fully CGI films, lies in how realistic they aim to make objects feel. VFX studios, such as Weta and MPC, use manipulated photos for the majority of their texturing, along with shaders and procedural textures, in order to keep things looking lifelike. Studios such as Pixar and Dreamworks however, use more hand-painted textures in order to allow them to represent their stylised animated worlds. Although, a common misconception is that they rarely use photos, which as Neil Blevins, Technical Director at Pixar Animation Studios, explains is far from the truth. “We use a lot of photo stuff over here at Pixar, as well as procedurals and hand painted materials, although the main focus with using photos is as a way to quickly provide us with texture, dirt, grief, etc, as opposed to VFX where the desired result is something that looks completely real… while our goal is to produce stuff thats a bit simpler and stylised, we definitely use a lot of photo textures to help us get there.”
Procedural texturing, a common way to texture in the industry, is a quick way to create textures within the 3D program rather than using ‘Photoshop’ or sourcing photo textures online. Mathematical formulas can generate shader textures, allowing for high resolutions and infinite variations, produced inside programs such as ‘Maya’s’ ‘3D Textures’, which includes options such as Wood, Rock, Marble and Leather. Coding can also be used to produce them, or 2D images can be referenced, called ‘refMaps’ or ‘Sampler’ images. The cons that lie within procedural texture however is that it does not allow for any artistic control, which is why a combination of hand painted and image texturing along with procedural texturing works well.
VFX has come a long way since the 90s and even the 2000s, and is now used in practically every film, along with even some television shows and advertisements. As such, texturing is also now used widely, with the feature films of today increasing in detail at an incredible rate, allowing for texture sizes to become almost unlimited. In 2012, Weta worked on the prequel to their earlier productions ‘The Lord of the Rings’, in the form of ‘The Hobbit’. VFX Supervisor, Eric Saindon, remarked that “The original Gollum model was around 5,000 polygons and the average texture map size was around 2k. Our new model [in ‘The Hobbit’] was closer to 1 million polygons and the average texture map was around 8k”. Later in 2013 texture artists were said to have spent up to 18 months painting the ship materials for the film ‘Gravity’ in order to produce the high level of realism that the film required, which was entirely made using VFX besides the lead role.
The Future
Currently, Pixar and Disney are in the process of experimentation with the way they create, and ultimately texture for their films. Starting with Pixar’s short film ‘La Luna’ in 2011, through to Disney’s ‘Paperman’ (2012) and later ‘Feast’ (2014), a combination of 2D and 3D animation working together is beginning to look like the next big thing. In ‘La Luna’, no procedural textures, shaders, or photos are used, and as such the film is made up entirely of hand-painted and man-made paintings. Furthermore, watercolour paintings scanned into the computer are made use of for the night sky background of the short. This already marked a change, opposing the usual clean and polished look of CGI animated features, instead opting for the organic and imperfect nature of painted textures. ‘Paperman’ and ‘Feast’ are both CG / Hand-drawn Hybrids which blend the two arts, and allow artists to paint or draw over the 3D models for the film, using an in-house custom software called ‘Meander’. Currently, the textures used on both shorts are very simple despite being hand painted, but are likely to become used more and more as Pixar and Disney continue working with them, and may eventually lead to a full length feature film using the technology. An example of this can be seen in the 2013 SIGGRAPH paper ‘Stylising Animation By Example’, in which experiments with the idea of painting over keyframes of animation is discussed, in which the frames are then blended to create a seamless digitally painted but CGI animated character. The paper also discusses overlaying CGI footage with textures such as traditional paintings, hatching, line drawings, and watercolour. While there are currently no announced films coming out of Pixar or Disney featuring this style of animation, it certainly seems on it’s way, giving their industry a very exciting and bright looking future that will feature a lot more hand-drawn textures and organic styles.
In march of this year, Pixar released their ‘RenderMan’ software online for free. Since ‘RenderMan’s’ birth back when ‘Luxo. Jr’ came out in 1986, the software has been used on almost every blockbuster CGI and VFX film to date, including all ‘Harry Potter’ (2001 - 2011) movies, ‘Jurassic Park’ (1993), ‘2012’ (2009), ‘Life of Pi’ (2012), ‘Brave’ (2012), ‘The Lego Movie’ (2014), and all of the ‘Marvel’ (2008 - 2015) films. This software that can simulate extremely realistic lighting for 3D objects is now available for anyone to use who has the know-how, with no time limits, watermarks or any other restrictions, other than for a commercial license of $500. At the same time, Pixar also released 128 of their classic tiling textures for free alongside ‘RenderMan’ under a creative commons license. It’s exciting to imagine what anyone with the skill to do so can now create practically for free using these new resources.
The future of VFX is also looking extremely promising these days, with recent blockbuster movie maker Marvel having a timeline of movies to make for their own ‘Cinematic Universe’ extending till around 2020. James Cameron’s ‘Avatar’, the movie that led Weta to create whole new programs and workflows to achieve the level of detail needed, is set for three sequels in 2017, 2018, and 2019, and it can only be imagined what level of intricacy will be required for these next level films, 10 years after the first one.
Conclusion
In conclusion, i’ve found that the “best current practices for the creation of asset textures in 3D film” is a combination of texturing with image-based, procedural, and hand-painted textures, in either a UV, UDIM, or projection texturing based software. There are many different ways to texture assets, and while the industry standard continues to be through the use of UV mapping and ‘Photoshop’, studios such as Disney and Pixar now rely completely on their own form of texturing within their in-house software ‘Ptex’. However, programs such as ‘Mari’ and ‘BodyPaint 3D’ are also extremely reasonable alternatives, and depending on the type of feature film being created and the style artists are going for, any one of these pipelines might be the most suited. While VFX companies rely mostly on photo manipulated images for their texturing work, CGI are following a different direction, instead focusing on the use of procedurals and hand-painted textures, although it would appear an even more organic feel is currently being sought after by the artists of Pixar. I now understand the process in which artists tackle texturing, and the many options and variety of ways to go about it. In my own opinion on the best product to use, I believe that while UV mapping is the current standard, UDIM-UVs will soon take over as they seem to be able to hold larger amounts of information, while maintaining a similar sort of workflow which will allow an easy switch over for artists use to UVs when this does eventually become the standard. Projection painting programs such as ‘Mari’ also have a bright future, although I feel that there are particular projects, such as the creatures on ‘Avatar’, that this pipeline will shine, and some in which it will not. In this case, while it is amazing at what it does, I do not think it will become industry standard. As for Disney’s ‘Ptex’, I believe that it is perfectly built for Disney and Pixar artists, and similar programs will eventually be picked up by studios such as DreamWorks and BlueSky, however I do not see this being used on VFX films or by many other studios. As the film industry continues to grow, and the need for CGI and VFX increases, I believe the level of detail will also need to expand, and with that the art of texturing, highlighting a promising future with these programs, and hopefully even more innovative ones to come.
Bibliography
The Art of Ratatouille by Karen Paik
"Despite the technological advances and hype surrounding projection texturing however, programs such as ‘Mari’, ‘BodyPaint 3D’, and ‘Ptex’ have not yet become industry standards, and UV mapping and ‘Photoshop’ continues to make up the majority of the texturing pipeline."
ReplyDeleteI'd disagree with your conclusion about Photoshop being the current industry standard. Mari is used as the main texturing tool at ILM, Pixar, Dreamworks, Sony Picture Images works, ImageEngine, Animal Logic, Framestore, MPC, Dneg, The Mill and many, many others. BodyPaint3D has for the most part, fallen out of use in the bulk of high end VFX studios.
Cool essay btw.
ReplyDeleteIf you have any questions about VFX texturing I can answer them for you.
ReplyDeleteThankyou! Since the writing of this essay which was some months ago it has become increasingly apparent to me the importance and now high industry use of Mari for texturing in the VFX industry. I haven't had the chance yet to delve into further research and study of it myself yet, but am very eager to do so.
ReplyDelete