FRELLED : A User's Guide
FRELLED is the FITS Realtime Explorer of Low Latency in Every Dimension, an astronomical data viewer designed for 3D FITS files. It's primarily aimed at visualising data cubes in realtime, interactive 3D, and is particularly geared toward HI and simulation data. This page serves as a user manual.
FRELLED is a set of Python scripts for Blender, an open-source 3D graphics creation suite. The scripts allow users to import 3-dimensional FITS files into Blender, where they can be viewed from any angle in realtime. Users can freely move the viewpoint to any position or rotation and the resulting image is instantly updated – there is no need to wait for any recalculation. The scripts are designed to work with observational HI data cubes and generic simulation data, but in principle there is no reason why any other 3D FITS files cannot be used.
Blender is first and foremost software for artists. It is designed for architectural, character, landscape and general-purpose modelling and rendering, and also game creation. It is not intended for scientific visualisation and has no inbuilt astronomical functions. However, being designed to allow users to visualise large amounts of 3-dimensional data means it has many features that standard mathematical visualisation tools lack. Fortunately, it also has a Python interpreter.
FRELLED’s main features are :
- 2D / 3D mode
- Allow the user to interactively generate mask regions within the data
- Masks can be freely toggled on and off
- Masks can be assigned names, which can be used to re-find them later
- Overlay SDSS RGB images
- Generate integrated flux maps on-the-fly
- Interactively generate contour plots and renzograms
- Generate output parameters for MIRIAD tasks : mbspect (can be run interactively), moment, mask
- Assign colours based on velocity as well as intensity
- Support for world coordinate systems
- Display vectors and n-body particles
The main motivational development behind FRELLED was that looking at 3D data in 3D is infinitely more awesome than looking at slices. However, it soon became obvious that the ability to interactively mask data made source extraction – literally – about 50x faster. Being able to quickly re-navigate to any particular source by inputting its name is a very simple feature, but one that also makes life a lot more bearable for anyone working with hundreds of potential detections within a data cube.
3D viewing may be more awesome, but 2D viewers can allow the user to find fainter sources more easily – noise emission along the line of sight can cause problems for a 3D viewer. FRELLED can also be set to operate as a conventional 2D viewer. Like kvis, it allows the user to switch between the various projections, while maintaining its other capabilities.
The FRELLED scripts allow Blender to access FITS files and are designed to make life as straightforward as possible for astronomers using them – but they do not change the underlying code of Blender in any way. Wherever possible I attempt to distinguish between what Blender is doing, and what new capability has been added by FRELLED.
Nah, I'm not going to bother with anything that corporate. All I ask is that if you're using FRELLED for science, consider citing the following paper :
The Arecibo Galaxy Environment Survey - VII. A dense filament with extremely long H I streams Taylor, R.; Minchin, R. F.; Herbst, H.; Davies, J. I.; Rodriguez, R.; Vazquez, C., 2014, MNRAS, 443, 2634
I've been developing FRELLED since 2012 and it's grown to over 11,000 lines of code, each one soaked in the blood and tears of my enemies, so citations are appreciated. See also the Feedback section.
Download FRELLED itself from the webpage. Currently FRELLED requires Blender 2.49 to run. This can be downloaded here. Blender 2.49 requires a Python installation. The scripts have been tested with Python 2.6.2, but in principle can operate with any Python 2 release, provided the following Python modules are installed :
In order to guarantee that Blender will correctly interface with Python (Blender has its own internal Python as well) it is recommended that you have a version of Python 2.6 installed, somewhere.
You also need a decent graphics card. To test if your machine has the necessary capabilities, download this test file, and open it with Blender (this file does not require any Python modules). If you see a galaxy (see image of NGC 5055 on the right) then you should not have any problems.
There is no installation process as such for FRELLED itself, you just need to download the files. However a very little initial organisation is needed to work with them properly.
First, save a master copy of the main files (FRELLED.blend and the accompanying Python scripts) wherever is convenient. The extra file “.b.blend” file should go in Blender’s .blender subdirectory (Windows) or the user’s home directory (Linux). Note that files with a name beginning with a ‘.’ are hidden on Linux.
Secondly, create a directory containing the FITS file you want to work with and a copy of the main FRELLED files. This directory will contain all the supplementary files that the software will create and work with. Potentially, this can include hundreds of image files running into gigabytes of disc space (though usually very much less).
FRELLED will automatically locate the FITS file contained within the directory within which it was placed. If you have multiple FITS files it will pick one at random, so it is best to work with one directory containing one FITS file. However if you wish, FRELLED allows the user to manually input the directory and/or filename. Although typing in the whole directory is a right pain, so don’t do that.
To start FRELLED : Navigate to the directory where you want to use FRELLED and enter the command, “blender FRELLED.blend”. Don’t open it from a GUI because error messages are printed to the terminal (on Windows a command prompt opens automatically).
A word of warning : Blender has an internal version of Python that operates independently of any other Python version installed. Calling some Python modules from within Blender (such as numpy) can sometimes fail spectacularly on Linux. FRELLED attempts to avoid this by running most of the FITS processing via external scripts. Provided you can run a Python script via the command, “python whatever.py”, all should be well (Blender must also be able to find and access the “os” module).
While other software may take several minutes to render a single view of a 3D file to an image, with FRELLED the process is instantaneous. This allows users to explore data sets in a completely new way.
Blender has no native capability to handle FITS files. Although it can work with relatively large data sets (on a good system, millions of data points can be handled in realtime) the size of many astronomical data prohibits the use of Blender's native internal data formats. The way to circumvent this issue is to import the FITS file as a sequence of images, for which Blender is far more optimised. In this way, data sets ~600^3 voxels can be accommodated.
FRELLED relies on the user having a capable graphics card. The downside to GPU rendering is that they usually have much less memory than ordinary RAM. This means that larger data sets must be broken into smaller subsets for use with FRELLED in 3D mode, though this is less important in 2D mode.
The software is distributed as the file FRELLED.blend which can be opened in Blender. The interface is configured to optimise Blender as a FITS viewer - none of its functionality has been disabled, just hidden. The ‘.b.blend’ file provides other colour schemes which are more suitable for a FITS viewer than Blender’s default.
In 3D mode, FRELLED displays FITS cubes volumetrically. In 2D mode it displays them as sequences of images, as with any conventional viewer such as kvis or ds9. Both modes have the same basic operational procedure :
1) The FITS cube is converted to a sequence of images. The data range and colour scale (linear or logarithmic) are controlled by the user. This must be repeated every time the user wants to change the data range or colour scale.
2) The images are imported into Blender. The images are retained in the directory in which they were created, they are not stored within Blender (though this is possible, see section Saving the settings).
3) Apart from simply viewing the data, FRELLED also serves as a GUI for several miriad tasks. The user can create objects within Blender which can serve to define parameters for tasks such as immask, mbspect, fits and moment. Additional features include the ability to perform a NED search and open the SDSS Navigate tool, without the need to type in coordinates. Generating moment maps, renzograms, and overlaying contour plots on optical images is also possible.
The following pages describe the usage of FRELLED as well as explaining how the programs work. As the operation of these is somewhat different to conventional viewing programs, users are strongly encouraged to actually read the manual at least once. Those who dive straight in will likely end up completely frelled.
Quick Start Guide
Rebellious users who decided to completely ignore what I just said should invoke the help of a magical moose. It’s the big button in the middle. You can’t miss it. The magical moose will automatically find a sensible data range to display, turn everything on that needs to be on, convert the FITS file to .png images, load those images, draw the axes, make the tea and do your laundry.
It won’t do as good a job as if the user carefully alters the settings themselves – moose are clumsy, because of the hooves - but it will be good enough for a first look. It’s also a useful way to find settings that can then be tweaked to give better results.
The moose is a slow, methodical animal and will load the entire cube. The faster alternative is to employ a princess, who will only load the shortest axis of the cube. This is much better if you want to see what your data looks like but you’re worried the world is about to end.
LESS QUICK START GUIDE : The Story Of Princess Olivia And The Magical Moose Whose Name No-One Could Quite Remember
Once upon a time, the fair Princess Olivia was bored and decided to look in her magic mirror. “Mirror, mirror, on the wall,” said the Princess, “who has the best HI data of them all ?”
“You, O Princess”, said the mirror, “but it would be even better if you looked at it with FRELLED.”
The Princess decided that this was a good idea and would go to the library to learn all about it. It was a long way to walk, but luckily a magical moose appeared from nowhere. That’s what magical mooses do.
“Climb aboard !” said the moose, “I’ll take you to the library in no time at all.” The moose ran to the library at an amazing speed. “Thanks, moose !” said the Princess, “ I don’t suppose you know where I could find out about FRELLED ?”
“Why yes,” said the moose, “for I am a magical moose.” The moose and the Princess sat down in front of one of the library’s many computers.
“First,” said the moose, “you should decide whether you want the data to be in 3D or 2D, and push the little blue button accordingly.”
The Princess sighed a disdainful sigh. “You can’t be a very magical moose,” she said. “Why would I want to look at data in 2D ? I am a Princess, you know !”
“Ah, quite,” said the moose, who magically blushed and shuffled his feet in an embarrassed manner. “How silly of me. Anyway, next you should press the ‘99’ button”.
“Why ?” asked Princess Olivia.
“This will work out how much of the data should be transparent,” said the moose, “and then automatically set the appropriate data value in the ‘maximum data value’ box. ‘99’ means that 99% of the data will be transparent, which usually looks quite nice.”
“How very convenient,” declared Princess Olivia in a regal manner. “Anything else ?”
“Yes,” said the moose, “You must choose which projections to export. Let’s try just the XY projection for now, it will be faster. Then you need to press the ‘Map 1’ button in the top part of the panel.”
The Princess did as she was bid, and gasped.
“Gasp !” said Princess Olivia, “I see that something else happened when I pressed map 1 !”
“Yes,” said the magical moose, “The top part controls how the data is converted, but it also sets defaults for how to display the data. That’s what the buttons at the bottom are for.”
The Princess was delighted that her regal fingers would be saved from the excessive manual labour of pressing more buttons than were absolutely necessary. “I suppose you are quite a magical moose, after all,” she said, grudgingly. “What’s next, moose ?”
“All you have to do now is press ‘Go !” said the magical moose.
“Alright,” said the Princess, “but will this take long ? I have to go and fight off an invading horde of goblins this afternoon.”
“Not at all !” said the moose, proudly, “The code is parallelised. It shouldn’t take more than a couple of minutes.” And indeed, in almost no time at all, the Princess marvelled to see a wonderful three-dimensional image appear on the screen. “Wow !” she exclaimed, “This is better than Avatar ! Especially since it doesn’t have any 9ft tall smurfs. But what about all these other buttons ?”
“There isn’t any time to explain. This is only a quick start guide,” said the moose, magically breaking the fourth wall. But he tipped his head and something white and fluttery fell from his antlers. “Try reading this manual. That should answer all your questions.”
Princess Olivia thought about this and adjusted her tiara in a haughty fashion. She wasn’t sure that Princesses were allowed to read manuals. Perhaps she could find a servant to read it for her. “Oh, alright,” she said. “Goodbye moose !”
The moose magically disappeared. Later on, Princess Olivia decided that fighting the goblins was too much effort. Instead she decided to teach them all about FRELLED. And they all analysed HI data ever after.
Opening FRELLED.blend should reveal something like the image on the right. The screen is divided into three main areas : 1) The 3D viewport, where the data will be displayed; 2) The Menu screen, where the user can control how they display and interact with the data; 3) The Timeline, used for special interactions with the data.e
The menus include help buttons which will load the appropriate section of this wiki.
The “Window selector” allows the user to switch between several different preset interfaces. Control panel allows the user to select different display colour schemes via the “Themes” tab (default is the grey background shown here, the "Visualisation" theme gives a black background and is often a better choice), and also alter the clipping of the displayed cubes via the “Clip alpha” slider in the “System & OpenGL” section. It is also recommended to first set the “Time Out” setting to zero to prevent Blender from dumping the imported images from video memory (unfortunately this setting cannot be saved, so this must be done every time the file is opened).
The default MainViewer window is optimised for running the scripts and viewing the data. 4Views has four preset viewports instead of just one, one for each projection plus one more at an oblique angle. Developer provides access to the FRELLED source code, with a slightly larger area for the menu code.
The menus section is where the various settings to convert and import the FITS file can be accessed. There are five available menus : Display, Analyse, Movies, Vectors and Particles. Green buttons at the bottom of each menu allow you to access the other (the “script selector” is only useful if you need to edit the source code). Should something go wrong (which is, obviously, impossible), the GUI menus will revert to the underlying Python code. To reactivate the GUI, press “Text” (next to the script selector) and then “Run Python Script”. Alternatively hover the mouse over the Python code and press ALT+P. Note that you cannot (and should not) access any other scripts until a cube has been loaded (even if that means just drawing its axes).
The “shader selector” determines how the images are rendered once they are imported. The default setting of “textured” shows them as actual images. “Solid” shows them as monochrome planes, “wireframe” shows the outline of the planes, “box” displays wire-cubes of the dimensions of the images, Note that pressing the Z key (with the mouse in the 3D viewport) will switch between solid and wireframe mode. Textured mode can be re-enabled using the GUI button, or by pressing ALT+Z.
Blender can place objects on any of 20 different layers. Objects can be on multiple layers simultaneously and multiple layers can be displayed. Normally FRELLED will automatically select the appropriate layer(s) to display.
The “timeline” area is only useful in 2D mode and for creating animations. In 2D mode, the short green line indicates which channel or slice of the cube is currently displayed, while “Start” and “End” indicate the slice or channel range of the cube in the currently-visible projection. Next to them is a box where (SHIFT+LMB) the precise channel or slice can be entered. When creating animations, the green line and the frame values allow control over which frame of the animation is currently shown. See section 2D Mode for more details.
A few other interface points to keep in mind :
- Any operation in Blender requires the mouse pointer to be in the appropriate window. For example, pressing ALT+P to run a Python script will have no effect if the pointer is in the 3D viewport.
- Selected objects are highlighted with an outline. ‘Active’ objects have a lighter outline. Objects are selected by right-clicking on or near them. The last selected object will be the active object. Press “A” to select or unselect all objects on the currently-visible layer(s).
- SHIFT+RMB on or near an object will unselect it if it is already selected. If it is not, it will be added to the selected objects.
- Any Blender window can be maximized with CRTL+↑. This can be useful to get the best possible view of the data, or if you want to edit the Python scripts.
- Progress status and error messages are displayed in Blender’s console window in Windows, or in the terminal from which Blender was run in Linux (if you open Blender via a GUI in Linux, you won’t get any progress or error messages, so don’t do that).
- The currently operating task is shown in the top banner which normally reads, “Blender 2.49” with a green background. If Blender freezes completely, it is probably loading images into video memory. If this takes more than a few minutes, give up.
A complete user’s guide to Blender can be found here.
A list of keyboard shortcuts is available here.
Converting the Data
To import the FITS file into FRELLED, use the default Display script. This is divided into two sections – the upper section controls how to convert the data, while the lower section controls how the data is displayed. Only limited control of the display is possible once the data is converted, so it’s important to choose good settings for this (otherwise it must be converted again, which can take several minutes).
FRELLED should automatically have located the FITS file within the directory containing FRELLED.blend – if so “Filename” will be replaced with the actual filename. If it did not, check you opened the correct copy of the .blend and that both it and the FITS file reside within the same directory. Make sure the file has the extension “.fits” and not “.fit” or “.FITS” or any other variant.
If you just press “Go !” at this stage, the axes of the cube will be drawn and nothing else will happen.
Although it’s preferable to work with only a single FITS file in a directory, it is possible to cope with multiple FITS in the same directory. If the wrong file has been found, you can simply type in the correct name (omitting the .fits extension). But, if there are no FITS files present, everything else will fail horribly and the world will explode.
You can also type in the full path name to the directory, although why you’d want to do that is a mystery.
The “XY / XZ / YZ” toggle buttons choose which projections of the cube to convert into .png images. Ideally all three should be selected, but it is often possible to work with just one or two, which can save time and memory. The projections have the same naming convention as kvis :
XY : RA – Declination
XZ : RA – Velocity
ZY : Declination – Velocity
“Norm” tells FRELLED to produce a normalised cube (see below), while enabling “Logs” will use a logarithmic intensity scale when producing the images. This requires the data values chosen to be positive and greater than zero.
The “90 / 95 / 99 / 100.0 / Cust.” buttons find the data values at 90, 95% etc. of the maximum value in the cube (100.0 is the default percentage used by “Cust.”). Pressing these buttons will find the value and set them in the data range boxes below (defaults 0.0 to 20.0). These are extremely important, especially in 3D mode. Data above the maximum value will be given uniform colour and transparency. For example, if you hit the 90% button, 10% of the data will appear to be solid white, and block the view of the rest of the data. 99% is generally a much better level to use. The lower data value (all points below which will appear completely transparent) is generally less important, unless all the values are offset from zero. Choosing sensible values to use can be a matter of guesswork and trial and error.
Useful trick : first load the cube using the "Princess Olivia" button. This will automatically select a 99% cut and find the shortest axis of the cube, thus minimizing the loading time. Then you can adjust the other settings and possibly get a better result, before loading the other projections with "Go !".
This section of the script controls precisely how to convert the cube into .png images. FRELLED allows the user to export two maps based on the intensity range of the data. The drop-down menus (defaults are both “Grey”) choose which colour schemes to use for each map. Next to them are the data range values to use (defaults 0.0 and 20.0), which can be independent for each map. The “Map 1” and “Map 2” toggles tell FRELLED which maps to export. Exporting just one map is easier, saves time, and is usually perfectly adequate. Using two maps can give better results and can be extremely useful for very complex data sets (see section Displaying the Data).
Pressing the toggles for any of the maps automatically sets defaults in the lower part of the screen for how the data is displayed - see section Displaying the Data.
The third drop-down menu and the corresponding toggle tell FRELLED to produce velocity maps. These vary the colour according to the velocity (or z-axis). For very extended sources filling much of the cube (e.g. galactic HI), these can help show the 3D structure of the data more clearly – e.g. GALFA-HI data, shown below.
The “2D” toggle tells FRELLED to use 2D mode. If not selected it will assume 3D mode. Note that this also affects the defaults in the lower part of the script – if you alter these, be aware that pressing “2D” again will restore their defaults.
"Center" is useful for navigation (see section Navigation in 3D Space) - it will place the 3D cursor at the center of the data cube.
“Go !” will do several things. The script will write the FITS header and the export settings to a text file, convert the cube into images (only if any projections are selected), draw the axes of the cube using world coordinates if possible, and – importantly – create several placeholder objects in Blender to tell it whether 3D or 2D mode should be used. These objects are essential to using FRELLED – the “Analyse” and other menus cannot be accessed until “Go !” has been pressed. More obviously, it will also import the images into FRELLED.
If your data is a funny shape you’ll get a pop-up asking if you want to stretch it to a cube (this only affects the display, not the data itself). If you enable it it will set the z-axis to have the same length as the x-axis. You can also stretch it by an additional numerical factor (default 1.0 in the pop-up), i.e. a factor of 0.5 will make the velocity axis half the length of the RA axis; a factor of 2.0 will make it twice as long.
If you selected “Norm”, then pressing “Go !” will also generate a whole new FITS file, with the data normalised to the maximum value in each channel. The minimum and maximum values must be between zero and one. For example, say you enter 0.25 and 0.99. In this case, in each channel, 25% of the data will be completely transparent and 1% will be completely opaque (ignoring NaNs). This means the contrast in each channel will be the same, regardless of the dynamic range.
This is a complete waste of time for cubes full of point sources, but can give dramatically better results for (say) galactic HI. Here the signal fills the entire sky, but in some channels the data is all very bright and in others very faint. Normalising the cube makes it possible to see both faint and bright structures, and is particularly useful if using a logarithmic scale gives poor results. This disadvantage is that the absolute intensity values are lost – this is good for enhancing structures, but not for comparing brightness.
If you reload a normalised cube (it will have been given the _norm extension) later, you should use values from 0.0 to 1.0 to re-create the display you had when creating it. In the above example, 25% of the values in each channel will be below 0.0 and 1% will be above 1.0. Unlike when creating the cube, when reloading it the min/max values are not constrained to be between 0.0 and 1.0
Displaying the Data
The lower section of the Display script is what controls precisely how the data is displayed. Unlike the settings in the upper section, values here can be altered without the need to re-export the images.
By default, pressing “Map 1” in the upper part will enable “Use 1”, “Alpha” and “Multiply” in this lower section. If nothing else is set, this will display the data as the sum of the intensity values along each line of sight. If you also enable “Colour” here, the display will give somewhat higher contrast, but is potentially more difficult to interpret.
Some colour schemes are suitable for use with a single map to control both colour and transparency (alpha), while others are only useful to control the colour (these are marked “2D” in the drop-down menus). This is because Blender generates transparency based on RGB colour, where black is totally transparent and white is totally opaque. So, only colour schemes which go from black to white can be used to control transparency and colour at the same time.
If you want the display to look prettier, the solution is to use two maps. By default, pressing “Map 2” in the upper part will enable “Use 2”, “Colour” and “Multiply” in the lower section. This means you can have map 1 controlling transparency, with another (nicer) colour scheme controlling the colour (see below for examples).
All of the colour schemes are linear, except for “High Contrast” which is a black-white colour scheme with R,G,B values v = f^5, where f = (flux - fluxmin)/ (fluxmax – fluxmin). This is useful when there are many more low flux values than high ones.
Both Map 1, Map 2 and velocity maps can be generated and displayed at different times. For example, you may first have exported a single intensity map, but then find that a velocity map would also be useful. In this case, you would then disable “Map 1” (not strictly necessary but this saves time - it prevents FRELLED from re-creating a map which already exists), but keep “Use 1” enabled (so that it will still be displayed). Velocity maps should then be enabled in both the upper and lower panels. Some examples are shown below.
“Apply” will attempt to update the settings for how the maps are displayed, provided they have already been generated and loaded. Another option is to use the “Load images” button, which re-imports the data into Blender. This also happens automatically if a map has not been previously displayed. “Load images” is useful if you know the .png images already exist and don’t need to re-export them; “Apply” is useful (and faster) if you just want to change how the images are displayed.If something goes wrong : you may see a big black monolith or, more likely, a milky-white transparent cube instead of your data. See the troubleshooting section.
Altering display once the data is loaded
“Update alpha” will set the transparency value. Zero will make everything transparent, 1.0 will make the data completely opaque at maximum intensity.
If no options are set in the lower panel, maps will be created but not displayed, which is normally pointless (though the axes will be displayed). FRELLED will run into difficulties if you try to display a map (lower panel) which has not already been created (upper panel).
Some other options are accessible via the Control Panel “System & OpenGL” tab, as shown in section Available Screens. “Clip Alpha” sets a threshold – any part of the images with an alpha value lower than that threshold are not shown. Higher alpha values are not affected. If you’re interested in bright emission, this is a very simple, instant and reliable way to remove fainter emission.
“Mipmaps”are a way to prevent the appearance of pixilation. This can give extremely pretty - but possibly misleading – results as it will appear as though the data has been sampled at an infinite number of points. They also require significant extra memory (~30 %) so are not enabled by default.
Using multiple FITS files
It’s even possible to trick FRELLED and get it to load a different FITS file for each map. Sounds crazy ? Normally yes, but for a simulation you may want to have the transparency controlled by the density and the colour showing (say) the temperature. First, produce Map 1,as normal. Then disable “Map 1”, and, externally, remove or rename the FITS file. Add the second file you want to display into the working directory, but give it the same name the first file had initially. Then produce and display Map 2 as normal, keeping “Use 1” enabled (this method is fine for rotation movies but not time series – see section The Fourth Dimension).
Saving the settings
While you can - and should - save a FRELLED file much like any other file (File -> Save As..; or hit F2), this will not remember your settings. It won't exactly forget them, either. Say you load in a cube displaying data between 0.0673 and 0.129 mJy. You save it and reload it (which is very much faster than having to re-import the data !). Your data will appear exactly as it did before, except that the values of the data will no longer appear in the menu boxes, nor will any of your other settings.
To preserve this information use the dialogue box near the bottom of the menu. This will save all your current settings to a filename of your choice (overwriting if it already exists). Of course you can also reload from a file in the same way.
If you save the .blend to a directory other than where you created it, the links to the images may be broken - i.e. when you open it you'll see nothing but a milky-white cube. If you need to share the file with your friends and family, use File -> External Data -> Pack into .blend file. This will save all the images into the .blend file itself, which enables you to show other people the data without actually giving them the FITS file. They will still need it if they want to run any of the analysis tasks, however.
Sometimes you might want to overlay one FITS file on another, e.g. multi-wavelength data, different polarisations, or multi-phase media in simulations. FRELLED allows you to display two different FITS files at the same time (see video for an example) with different colour schemes and transparencies for each. More than two components are possible in principle, but not yet supported. There are also potential memory limitations since displaying two data sets requires more GPU memory and reduces the framerate (potentially a solvable problem in the fullness of time, however).
The procedure to load multiple FITS files is straightforward. In the file selector box, enter the name of the first file and give it the extension ".fits1". Choose whatever colour scheme and data range you want (as above) and press "Go !", just like loading in any other file. Then load in the second file by giving it the extension ".fits2" - and remember to choose a different colour scheme !!
Note that you only enter the .fits1 and .fits2 extensions in the file selector box in FRELLED - do not rename the files themselves !
It's probably a good idea to load the files separately first to get a good idea of what display settings look best.
If you need to change the transparency level of a particular data set, just give the appropriate filename with the .fits1 or .fits2 extension and "Update Alpha" will only affect that component.
It's up to you to ensure that the files have exactly the same world and pixel coordinates. They must have exactly the same dimensions or I have no idea what will happen.
Once the data cube has loaded, you can start to look around. The viewpoint controls are as follows :
Pan : SHIFT+MMB
Rotate : MMB
Zoom : Wheel
Place cursor : LMB
Center view on cursor : C
You can also pan left and right with the mouse wheel if you hold down CTRL, and up and down by holding SHIFT. Note that this refers, for example, to up and down in Blender’s viewport – depending on how you have rotated the view, this does not necessarily correspond to north and south.
The 3D cursor serves as a marker. If it’s in the way, simply move it somewhere else by left clicking. Centering the view on the cursor also changes the pivot point (around which the view rotates) to its location. If you get lost, you can re-center the cursor to the center of the cube with the “Center” button.
Navigation is also possible with the numeric keypad :
Zoom : + / -
View XY projection : 1
View XZ projection : 7
View ZY projection : 3
Toggle perspective : 5
Rotate up : 8
Rotate down : 2
Rotate left : 4
Rotate right : 6
Rotation via the keypad is done in constant increments of 5 degrees. This can be changed in the Control Panel view in the section “View & Controls”. You can look along the opposite direction of any projection with CRTL+N, where N is 1,3 or 7.
Some limited navigation is also possible from the Analyse menu.
FRELLED supports some basic data analysis tasks directly and can enable more complex tasks in external programs more easily. The key feature is that the user can interactively generate "region masks" around chosen parts of their data, to mask and catalogue data, produce contour maps, query the SDSS and NED, and (via other programs) do just about anything else that requires defining a volume. Of course, this is also possible in other programs - the advantage here is that this can be done interactively through a visual interface. This is far more rapid than defining coordinates numerically. When precision is required, it's also possible to enter numerical values.
To generate a region, find something interesting in the data and place the cursor in the center. Ideally, use all 3 projections to accurately determine the where the center is (see section Navigation in 3D Space). Toggle between orthographic and perspective view mode, if necessary, with 5 on the numeric keypad. Choose the name you want to give the source in the “Name” box. The counter should ordinarily be left at zero.
Once you have placed the cursor where you want it, press “Add” to create a region. This will appear as a black cube, which FRELLED will assign the name you have chosen. You can change the size of this region with the “S” key in the 3D view and either use the mouse to set the scale, or type in a scaling factor with the keyboard. To limit the scaling to a particular axis, press X, Y or Z as required. When you are happy with the scaling, press the LMB or the return key to accept it, or RMB to cancel and start over. You can also adjust the position of the region with the G (for “grab”) key.
As a more readable list of useful keystrokes, I strongly recommend learning the following :
G : Grab a region and move it around.
S : Scale a region.
R : Rotate a region. Avoid this - most analysis tasks ignore it ! See below for details.
X / Y / Z : While the above transforms are active, constrain them to the specified axis. Conventionally, X is RA, Z is Dec and Y is velocity.
CTRL : While the above transformations are active, constrain changes to be in integers (e.g. move a region exactly 1 pixel at a time in the mouse direction).
SHIFT : While the transforms are active, makes them much less sensitive to mouse movement. Useful for more precise adjustments.
LMB : Accept the current active transformation.
RMB : Cancel the current active transformation and return the active region object to its last accepted state.
If you're a jerk - or just careless - and create a region which extends outside the cube, the various analysis tasks described below will truncate that region if necessary.
The drop-down menu with the default setting "Cube" allows you to select what sort of region shape you want. In general you should leave this on cube as that's what most of the analysis routines assume.
Naming and finding regions
The name of the region created is displayed in the lower-left corner of the viewport (e.g. “Name_001”). The numeric suffix will be automatically updated as you add more regions. To select a previously generated region, type its number into the counter box (and change the name if necessary) and press “Select”. The region will then be highlighted with a blue outline. If the counter is set at zero, all objects with the entered prefix will be selected. You can then center the view on that region by pressing “.” on the numeric keypad (which will also automatically adjust the zoom), or via SHIFT+S, choose “Cursor->Selection” and then pressing C to center the view on the cursor as normal.
If you add more regions while the counter is not zero, the names of the new regions will be automatically adjusted if there is a conflict with existing objects. If, say the count is 5, but object “Name_005” already exists, the object will be given the next available name. If however, object 5 does not exist, then the program will assume the user intended to start from 5 and continue naming sources working upwards from 5 (this is useful if you already have another source catalogue and want a consistent naming scheme).
Blender does not allow objects to have names longer than 21 characters. To allow the numerical suffix, the names of regions are limited by FRELLED to be 13 characters long. This theoretically allows the user to generate up to 10 million regions, which is overkill for the foreseeable future. Names cannot follow the widely-used "JHHMMSS+DDMMSS" format, and a jolly good thing that is too. Such names are unpronounceable and lead to many papers being unreadable.
Selecting regions interactively and controlling visibility
Sometimes it's more useful to have a region displayed as a wireframe, sometimes as an opaque mask. Both display modes can be toggled using the "Wire" and "Solid" buttons (these will act on all selected regions). It is not currently possible to hide a region completely. "Solid" mode displays the regions as opaque black, changing the colour is not currently possible.
Sometimes you may want to select a region without knowing its name. In this case the region can be selected by right-clicking near it (or SHIFT+RMB if you want currently selected regions to remain selected). However, because the FITS file itself is displayed using Blender's internal objects, this can often result in selecting part of the FITS file instead. To avoid this, use the blue "FITS" toggle button to temporarily disable the view of the FITS file.
Creating source catalogues
Once you’re happy with / bored / sick of creating the source catalogue you can export it for use with whatever programs you need to do actual science with. “Export” will save the selected regions in a text file with a format that can be read by MIRIAD, as well as a second file for importing them into FRELLED with the “Import” button.
Regions are saved in two formats : 1) “BlenderMask.txt” containing the regions in pixel coordinates, which can be read in by FRELLED; 2) “WCSMask.txt” containing the regions in WCS format. Both are saved unless your data doesn’t have a WCS, in which case only “BlenderMask.txt” will be created. If you have both, pressing “Import” will open a pop-up allowing you to choose which file you want to import from. Be aware :
- "BlenderMask.txt” contains everything about the imported regions, including spatial width, but in pixel coordinates. Importing this file into a data set from another part of the sky won’t give you meaningful results.
- "WCSMask.txt” contains the data in WCS, but doesn’t (currently) save the spatial size of the regions.
- Neither file saves the region shape – cuboids are assumed. If you need to save weird-shaped or rotated regions, you need to save the Blender file (see section Saving the settings).
If you want to import external catalogues using the WCS file, the format is (whitespace separated) :
RA (degrees) Dec (degrees) Velocity(km/s) Velocity width (km/s)
The "BlenderMask" file has the following format (whitespace separated) :
Name X Z Y XW ZW YW
Where X, Y, Z are pixel positions and XW, ZW, YW are pixel widths.
Currently the only way to preserve region shape and rotation is by saving the .blend. Exporting regions destroys this information !
Analysing the data with MIRIAD
The “mbspect” button acts as a GUI for the MIRIAD task mbspect, which is useful for quantifying the parameters of HI detections. I’m going to assume the user is already familiar with msbpect, if not, you probably don't need it (or you’ll have to Google it). Some parameters here are still hard-coded to assume the user is looking at HI data.
With only one region selected, “mbspect” will first check that the command “miriad” will actually start MIRIAD. If it doesn’t, that’s your problem. If it does, it will check if a MIRIAD file exists (looking for the .mir extension). If it doesn’t, it will then run the MIRIAD "fits" task to convert the FITS file you’re working with into MIRIAD’s own format (the original file is preserved). Note that if you already have a MIRIAD file – even if it’s the wrong one – FRELLED will use that rather than converting it. Be very careful when there are multiple FITS files in the same directory !
Once that’s done, it will use the selected (target) region as the profile in mbspect, displaying the extracted spectrum in an XSERVE window. Masks will be generated using the selected object plus any other regions which overlap the target on the sky. The source is assumed to be a point and the “width” parameter set to be the lowest odd number larger than the beam in pixels. It will attempt to do position fitting, but if this fails, it will automatically re-run with position fitting disabled. A comment will be printed on the spectrum to let you know that position fitting failed. Note that sometimes MIRIAD thinks position fitting succeeded, when in fact the fitted position can be miles off, so always compare the input and fitted positions. This is good advice when using mbspect in any case.
This interactive mode is intended to provide a useful first-look at the data. For a more detailed analysis where other parameters may need to be altered, the script also writes the input parameters to a series of text files, named according to the selected regions (placed in the mbspect subdirectory), e.g. :
in=Virgo1_SN.mir coord=12:13:52.16,07:12:15 width=17.0 xaxis=FELO yaxis=point xrange=122.114274114,3006.84387422 order=-1 options=posfit,measure mask=2422.13690635,2902.45355412 profile=2474.51492181,2849.92713676 device=/XSERVE
Similarly, “moment” generate inputs suitable for the miriad moment task (though it is almost redundant, see below). Usefully, “mbspect” also generates “QuickCat.txt”, containing only the name, RA, Dec and velocity of the selected objects.
Querying NED and the SDSS
It is often useful to be able to quickly search other data sources at the coordinates of an HI detection. FRELLED enables queries of NED and the SDSS through the "NED" and SDSS buttons. The "NED" button conducts a near-position NED query at the center of the selected object (i.e. region) with a 1.75' radius. Redshift is ignored since it's possible that any optical (or other wavelength) counterpart will not have a redshift measurement. If multiple objects are selected, an HTML webpage will be generated with links to queries for each object. In any case, a web browser will be automatically opened to display the results of the query.
The "SDSS" button is very similar, opening an SDSS dr10 navigate window set to the position of the selected object (or an HTML link page if multiple objects are selected). Unlike the "NED" button, the "SDSS" button tries to set a sensible field of view using the beam size. This works well for single dish data, but is very silly for interferometers, something which will be addressed in a future release.
Overlaying SDSS optical data
The “RGBs” button is a more fun way of querying the SDSS – rather than opening a browser, the RGB images are downloaded and displayed directly in FRELLED. The images are set to have the same field of view as the regions. The native 0.3961” resolution of the SDSS is used, unless that means the image would exceed a maximum 2048x2048 pixel size. In that case the resolution is altered so that the field of view remains correct.
It is not normally a good idea to display both the HI and optical data together. The view of the FITS file can be toggled on or off using the blue "FITS" button (see section Controlling visibility of maps for more details).
The RGB images are always oriented so as to appear face-on in the XY projection.
Contour plots and renzograms
The contour-plotting routines assume the regions are non-rotated cuboids. The three text boxes set the minimum, step and maximum contour levels. If “Norm” is toggled, values are normalised to the maximum (set to 1.0) in that region. Note that when requesting any map, the region size is first corrected to be a positive integer, preventing errors when trying to integrate over non-integer pixel values.
The “Mesh” toggle tells FRELLED to import the contours using Blender’s own internal format. This generally gives nicer results, however for very complex data this may cause memory issues. If in doubt, leave if off. Standard .png images will be used instead.
“Conts” generates contours for the integrated flux in each selected region, while “Renzo”generates renzograms (channel maps with the contours at each channel being assigned a different colour). The colours generated depend on the colour scheme and space selected from the drop-down menus.
“Local” means that the velocity (z-axis) range of each selected region will be used to generate colours independently. For contours this just sets all the contours to be white. For renzograms, it means that each renzogram will have colours ranging from (if “heat” is used) yellow to red. “Global” forces this colour range to be scaled along the velocity axis of the entire cube. For instance, red contours will occur at exactly the same velocity in every source, regardless of how large a velocity range the regions span. Similarly, “Selected” scales the colour range to the velocity extent defined by the selected regions. These options are intended for creating large numbers of contour maps using regions that may be spatially widely separated.
Mapping and measuring the total flux, and region statistics
“Flx Map” generates flux maps (greyscale only, regardless of any colour scheme selected) for the selected regions. As with the other maps it assumes non-rotated cuboid regions. At present only integrated flux maps (i.e. moment 0) are possible. Extending this to other moments should not be difficult.
“Flux” (upper part of the menu) sums the total flux in a selected region, printing the raw sum and values corrected for the velocity resolution and beam size (tophat and Gaussian). This works for regions of any shape and/or rotation. Including monkeys. It does not produce a map, only numerical values returned to the terminal. If the data does not have a WCS only the first quantity ("Sum of values"), which is uncorrected for the beam, will be relevant. It also produces the file, “VolumeCoords.txt”, containing the xyz coordinates located within the region. This means the user can generate weird, arbitrary-shaped volumes with FRELLED and then use them in other programs, if necessary.
"Flux" also calculates some basic statistics within the selected region (minimum, maximum, mean, median and standard deviation). Like the sum of values they are computer for regions of arbitrary shape, but no beam correction is used.
For the purposes of generating a nice plot to show people, the green "Axes" button generates axes around the selected region (but only one at a time - not multiple regions at once !). As with the axes for the whole cube they will use world coordinates if possible, otherwise default to pixel values.
At present the axes-plotting routines are robust to cubes down to a few minutes of arc/time and 1 km/s of velocity. Future releases will allow for smaller cubes to use world coordinates.
Controlling visibility of maps and contour plots and things
As already mentioned, it's often useful to disable the view of the FITS file. The visibility of the FITS file and other components can be changed by a series of toggle buttons. The toggles affect all maps - i.e. turn off RGBs and all SDSS maps will be hidden from view. You can't hide a specific map.
However, the "Select" and "Delete" buttons are linked to the region name box at the top of the script. If, for instance, none of the toggles are active and the user presses select, the region matching the specified name will be selected (identical to the "Select" button at the top of the script). But if the "RGB" toggle is pushed, then the SDSS map generated from that region will be selected, if it exists (or from all regions with that name prefix if the counter is set to zero).
You can move the maps and contours around if you want, but if you do, the only way to move them back again is to regenerate them.
It can be useful to select the axes of the main cube. To do this, enter the name "FitsCube" in the source name box, then enable the axes toggle and hit select.
The "Delete" button has a special behaviour. While you can remove anything in Blender by hitting the delete key on the keyboard, do not do this. Objects are not truly deleted until the file is closed and re-opened. Until then, they still exist in memory - they are simply not displayed. Consequently, if you were to delete, say, region "Galaxy_003", "Galaxy_003" would still exist. If you tried to create a replacement, Blender would name it "Galaxy_003.001", which causes horrendous conflicts and probably the UN will become involved. The "Delete" button also renames objects so that naming conflicts never occur.
If you move a region, any associated maps don't get moved along with it automatically. You'll have to manually select them as well, or (better) regenerate them after you place the region in its new position.
The “navigation” section is hopefully self-explanatory. The green “XY / XZ / ZY” buttons control which projection to display – i.e. the viewpoint’s orientation. They have extra functionality in 2D mode (see section Regions in 2D). They operate completely independently of the projection buttons in the Display menu, so if you try to display the XY projection but didn't actually import it, you won't see anything.
The "centre cursor" button should help if you get lost - it places the 3D cursor at the centre of the cube. Remember you can press "C" in the 3D window to re-centre the view on the cursor.
"Peak cursor" is a bit more complicated. What it does is place the cursor at the coordinates of the maximum intensity along the line of sight, which is useful for finding sources. For example if you view the XY plane (RA-Dec) and you left click to place the cursor on a source, you have no idea of the Z coordinate (velocity) of the source, and no direct control over the cursor's Z position. In this case the "peak cursor" button looks along the Z-axis and places the cursor at the channel with the maximum intensity at those XY coordinates. Chances are that will correspond pretty well to the source you spotted in the XY view. There are a few caveats to this (to be fixed in a future version) :
- It doesn't look along the exact line of sight because that's difficult to code. Instead it looks along the nearest major projection that was actually imported. So if you're not quite face-on but still viewing the XY projection, it will look a long the Z-axis. If you didn't actually import the XY projection but are still at a viewing angle where you should see it, expect some strange results ! Ideally, import all three projections.
- It doesn't take into account any mask regions created, unless you actually masked them in the FITS file itself. So if you've already hidden one source and there was another at those spatial coordinates, say, it might centre the cursor on that source if it was brighter than any other visible sources in that region. The idea is just to make it easier for users to get used to the navigation interface, though future versions will probably account for mask objects.
- It often generates "invalid value" warning messages in the terminal, which as far as I can tell are utterly useless and can safely be ignored.
The 2D mode of FRELLED is very similar in operation to the 3D mode. The only difference to the export script is that it names the images differently, so that both 2D and 3D versions of the data can (in principle) co-exist in the same directory. The import procedure in 2D mode recognises the name difference automatically.
One thing to be aware of is that after loading a cube in 2D mode, you should set which projection you want to view using the “Analyse” menu – the default is set up for a nice 3D view of the cube, which gives a strange appearance in 2D.Although this means the user is forced to re-export all of the images for the 2D viewer, this is desirable since the main advantage of a 2D viewer is that lower flux levels can be included – allowing the user to search for fainter sources, without being confused by intervening noise. Therefore the user will probably want to alter the exported data range in any case.
To allow the user to pan through the cube, FRELL-2D employs Blender’s animation system. Blender’s timeline is displayed in the bottom of the 3D view. To pan through the cube, use the left and right keyboard cursor keys to advance/retreat by one frame. The up and down cursor keys advance and retreat by ten frames. The “Play” button in the timeline (or pressing ALT+A in the 3D window) will continually advance the frame until the end frame is reached (or the user presses escape), at which point the view is reset to frame one. The start and end frames are automatically set when using the XY/XZ/ZY buttons in the “Analyse” menu. The “Playback” button lets you set a delay between frames.
Regions in 2D
There are two other subtle features to the 2D version. Firstly, if you have just created a region and change the visible projection (using the buttons in the GUI menu – not using the numeric keypad), the frame will be automatically set so that the slice displayed corresponds to the center of the object. For example, suppose you create a region in the XZ projection centered on channel 73. You scale the object along the velocity axis, but then you want to change to the XY view to accurately set its spatial size. In kvis, if you change the view, the frame resets to 0. Not so in FRELL2D. The frame is automatically set to 73 when you change to the XY view so you don’t have to trawl through the cube to find it again.
Secondly, it is worth noting that the mask regions are automatically rendered invisible if the frame displayed does not include them. For instance, suppose the region described above spans the channels 63-83. Then in the XY view, it will only be visible between frames 63-83.
If you create regions in 2D or 3D mode, they will appear in the other mode automatically. They are linked - changes made in 2D also occur in 3D mode, and vice-versa.
The Fourth Dimension (or how to impress people in presentations)
Blender can create arbitrarily complicated camera animations, but I don’t have time to explain how. Instead, here’s a short guide to using FRELLED to create turntable animations and animate a series of FITS files.
All animation in Blender requires a virtual camera from which the viewpoint can be rendered. The “Movies” menu provides control over this camera (the little pyramid structure in the image on the right) and an object called an empty (the set of arrows) which acts as its pivot point. Actually there are two empties, one which the script controls to make the camera smoothly rotate, and the larger one which the user can control (e.g. for moving the center of the field of view, or for altering which axis the camera rotates around).
The camera setup in the figure on the right can be accessed with the “enable” button. If any part of this has somehow been deleted, the whole thing will be automatically recreated. The “Alt” button controls the altitude of the camera above the empty; “Az” controls its azimuth. Amazing. "FOV" controls the field of view in data pixels – this is automatically set to something sensible by the script. Setting it back to 999 will return it to the default.
Creating a turntable movie is as simple as setting the number of frames it takes for the camera to complete one revolution (setting “Frames” to 10 or less will cause any existing animation to be removed). To preview this, press “CmView” to see the view from the camera, then ALT+A in the 3D view. Press escape to stop playback. You will probably need to import at least two projections to avoid the FITS file disappearing from certain angles.
You might want to alter the axis around which the view rotates. To do this, select the pivot point using the “Position” toggle button. Just like with positioning regions, you can then rotate this using the “R” key in the 3D view. LMB accepts the chosen rotation, RMB cancels. ALT+R restores the default. “G” will grab the empty to translate its position. While rotating or moving the empty, pressing X, Y or Z will constrain the transform to that axis.
Remember, if you move the camera or the empties manually, you must save the .blend file. Whenever you re-enter the number of frames, any manual edits you've done will be lost ! Sorry about that.
Once everything is tickety-boo, you’ll probably want to save the images. The defaults, “X : 500; Y : 500” control the size of the images in pixels. Larger images use more memory, values above 2000 are not recommended. “OSA” turns on anti-aliasing (removes jagged edges) and can be slow. “T :4” sets the number of threads for rendering. Using one less than the number of (virtual) cores will prevent your machine becoming unusable during rendering. “Still” will render a still image at the current frame, which can be saved (once generated) with F3, or via the File-> Save Rendered Image menu. “Animation” renders the entire sequence of images, beginning at the “Start” value, “Play” plays back those images as a movie.
Note that the “Start” value does not affect when the camera starts rotating, only the point in the animation from which rendering begins. This means that if rendering is interrupted you can resume later without having to start from the beginning. The rendered images are saved in the "RotationMovie" subdirectory.
Converting the image sequence to a movie file is left as an exercise for the reader.
Time series animations
Animating a sequence of FITS files is relatively straightforward, provided they all have the same dimensions. In fact, you can animate a whole batch of time series at once. This is a simple procedure in FRELLED, but it does require the files be organised in a very specific way.
Each time series of FITS files should be stored in a separate directory. It doesn't matter where this directory is. In addition, within each directory there must be a file called exactly, "FitsList.txt" which contains a list of the FITS file in that directory you want rendered in their correct order (Python's automatic file listing function being lousy). On Linux this is a straightforward matter of entering :
ls *.fits > FitsList.txt
to a console in the correct directory, assuming the files are easily sorted.
You then need another file that lists those directories. This file can be called anything you like but it must have a .txt extension and be in the FRELLED directory you're working in.
Once that's all done, and you've setup the camera as described above, it's simply a matter of entering that filename in the file box (include the .txt extension !) and pressing "Go !". With the .txt extension detected, the standard behaviour of FRELLED is over-ridden. Instead it will now load in each file and render them from the camera, saving the image to the appropriate directory (given the name "Render_XXX.png" where XXX is the frame number). At present, each file will be rendered using whatever data min and max values you entered - you cannot have it automatically choose a different cut for each file. So it's a good idea to try loading in a few different files to make sure you've picked sensible values.
Unless you’ve also set the camera up to rotate, you should only have one projection selected. Rendering 100 FITS files each of 100^3 voxels from all 3 projections would require rendering 30,000 images (60,000 if you use two maps), and that would be pretty stupid. Even rendering just one projection still means 10,000 images, so don’t expect this to be quick.
The subdirectories holding the images (i.e. XY / XZ / ZY) are emptied once each FITS file has been rendered, but it might be a good idea to empty your trash folder after rending a FITS sequence !
Sometimes it’s useful to be able to draw a lot of pointy arrows on things, and that’s where this script comes in. It’s designed to work with velocity vectors from simulation data, but it should be simple to use any other vectors from any data set.
The vectors must be loaded from a text file of format x, y, z, vx, vy, vz, density, temperature. Density and temperature aren’t really necessary, but apparently they make for useful thresholds if you want to avoid displaying vectors all over the place (if you don’t have anything equivalent, just make columns of zeros in the file). The coordinates and velocities should be in pixel values – though the absolute values of the velocities don’t matter. At present the script can only show the direction of the vectors, not their magnitudes.
“Multiply size of vectors” does exactly that, to their tails. “Size of arrowhead” controls the absolute size of the pointy bit (yeah I know, it’s inconsistent – but whatcha gonna do). “Percentage of data shown” controls that parameter by skipping lines in the input file – it’s set to 1 by default for fast loading. Things will start to get slow if you try and load in more than a few tens of thousands of vectors.
The colour boxes open colour pickers where you can pick the colour of the heads and tails of the vectors. Only the heads of the vectors will show any colour in the realtime view.
Currently any changes (even just to colour) require re-loading the vectors. Also you won’t see them unless you press “enable”, which isn’t set automatically when you press “go”. Sorry about that.
My goodness me, that's a lot of buttons, isn't it ? Fortunately most of them are the same so it's not as many as it first looks.
There are almost certainly a bajillion better n-body particle viewers out there, but it might be useful to display them in FRELLED alongside (presumably) the gas data. This is a hack of an old Blender particle viewer into FRELLED. It allows you to display up to three different particle components, tentatively labelled as “Gas”, “Stars”, and “Dark Matter” (we’ll get to “Halos” in a minute). I have no idea why you’d want to view gas particles in addition to the gridded gas, but there we are.
All three particles are handled in the same way. The first line of the file should state the total number of particles, the second should have the number of frames, and subsequent lines should have the format x, y, z pixel coordinates. “Enable” means that this component will be loaded when “Go !” is pressed – it also checks that the specified file exists and retrieves the number of particles and frames. “Copy” copies the number of frames to the other components.
“Snapshots” disables animation of the particles and lets you see multiple frames at once – set it to 10 and every 10th frame will be shown at the same time. Obviously this is only useful for situations where the whole collection of particles moves significantly between frames, though setting it to 1 is useful to show particle trajectories. This is also useful if you just want to load a single frame for an animation (otherwise the particles will only be visible on frame 1).
The right-hand section controls the material properties of each component (for the rendered display). The coloured boxes open colour pickers, the other settings should be obvious. “Defaults” restores the default settings, “Apply” sets the chosen values (if you don’t do this, your settings will be ignored in the render). Note that the defaults are currently lousy. I only have particle data in physical positions (not pixels) to play with, and it’s entirely possible they won’t show up and all when rendered. Expect a lot of trial and error here.
The “Halos” component works differently. Here particles are loaded not as points but as spheres, which can have different radii (this is required in the file). The was originally used to represent the virial radius of galaxies in a cluster. I have no idea if there’s a more general purpose use for it, but it’s here anyway.
Importing Data from Other Sources
FRELLED is designed for HI data cubes and numerical simulations in mind. But there's really no reason it can't load in data from other sources, though this does require some trickery.
For example, it is possible to use kvis to export the images instead of using the internal conversion script. Kvis offers a wider choice of colour schemes and more interactive control of the precise range of data values displayed. The “Export .ppm data movie” option easily allows a cube to be exported. But, asides from being inconvenient, this option only works reliably for the XY projection – bugs in the software often crash it if other projections are selected. Also Blender (like most other programs) cannot read the .ppm files which kvis produces, so they must be converted into a more sensible format first.
On Linux, this can be theoretically done with the “convert” command. However, despite considerable effort, I have never been able to get “convert” to do a damn thing, and would really appreciate it if someone can tell me what I’m doing wrong.
On Windows, there is a free program with the unlikely title of “Pearl Mountain Image Resizer” which is very easy to use and can quickly convert hundreds of files into something more useful, like .png. It is a shareware program so will occasionally ask you to register it, but fortunately these pleas for money are kept to a tolerable minimum.
The only requirement for importing images into FRELLED is that they follow the naming convention, “FILENAME_XY_N.png” syntax, where N ranges from 000-999 (must include the trailing zeros). In principle it is possible to import any set of images, astronomical or otherwise, provided this naming system is adopted. However, this requires some tweaking of the code to get it work – contact me for details. It’s pretty straightforward – here’s an MRI scan of a banana flower and a scan of the human body, for examples.
Troubleshooting and FAQ
I keep getting “Out of memory” errors.
This is most likely to happen if you’re trying to import a large data set and don’t have very much RAM. A particular niggle is that if you are using 32-bit Python, you may not be able to use all of your available memory. 64-bit Python helps avoid this issue. You should also consider splitting your data into subsets and import them into separate files. FRELLED does not yet support internally specifying subset regions, so you’ll have to do this externally in whatever program you prefer
Blender keeps crashing.
It’s possible to have plenty of RAM but not much video card memory. In this case, Blender may well be able to export/import the data, but then collapse in an unsightly fashion when it tries to display the data (this is most likely to occur when changing the view orientation, forcing the images from other projections to be displayed).
One way around this is to change the “Time Out” value (see section Available screens) to something small but non-zero. This will allow Blender to free unused images from video memory, allowing larger data sets to be viewed. However it will also require all of the images to be reloaded when the view changes to a different projection, causing an unpleasant slowdown (this should not be a big deal for the 2D version).
Future versions of FRELLED will allow the user to import only every nth channel (or slice) from a data cube, reducing the burden on the video card. For now, consider regridding the data in an external program, or splitting it into subsets.
Also, remember that there’s no compulsion to import all 3 projections of a data set. Often a single projection will give a very useful quick-look, and 2 projections are usually sufficient for more detailed examination.
Why can’t I select regions I’ve already created ?
If you’ve imported sources from another file, they will be given the same name you assigned them in that file. Make sure you edit the “Name” parameter in the text box before trying to select them. If you can’t remember what the sources are called, try right-clicking to select one of them and its name will be shown in the lower-left corner of the 3D view (this may be easier in bounding box mode, see section Blender functionality). You can also check the “BlenderMask.txt” file.
I can’t see some sources in the 3D view.
As noted in section Converting the Data, displaying the full data cube requires that most of the low flux values be clipped or at least highly suppressed, otherwise the user will only see an opaque cube full of noise. Even with a cube in S/N format, some sources are so faint that they are not visible without also including a lot of noise. This is a flaw inherent in displaying the sum of the values along the line of sight. Currently the Blender realtime engine does not support displaying the maximum line-of-sight values, which can give superior results.
The 3D view is useful as a first look at the data and for studying extended sources. However, to see faint objects, you’ll need to go to the 2D view.
I see a big black scary monolith instead of the FITS file...
This is most likely impossible in current versions of FRELLD, but just in case :
Most likely one of two things : you’re trying to load in images which don’t exist, or you have matplotlib issues (in which case see next question). For example, suppose you open a FITS file for the very first time and set “Use 1” without “Map 1”. This means the script is trying to load an intensity map which doesn’t yet exist. You need to set “Map 1” on and try again.
If you think you already produced the images, it could be that you’re in 2D mode when you previously produced the images in 3D mode (or vice-versa). Just try changing the mode and see what happens.
...and/or I get an error “You must first define an image, e.g. with imshow” when exporting images
Oh noes ! This likely means you’re using matplotlib 1.1.1, where the developers decided to break everything. If you can’t update your matplotlib version, there is a simple fix. In a Python shell type:
>>> import matplotlib >>> matplotlib.__file__
This will tell you the directory where there is a file “pyplot.py”. Open this file in a text editor. Find and comment out (add a # to the start of the line) the following lines, then try again :
if mappable is None: raise RuntimeError('You must first define an image, eg with imshow') if im is None: raise RuntimeError('You must first define an image, eg with imshow') else: raise RuntimeError('You must first define an image, eg with imshow')
I get the error : “ImportError: /home/rhys/Software/miriad/linux64/lib/libgfortran.so.3: version `GFORTRAN_1.4' not found (required by /usr/lib/liblapack.so.3)”
Urgh. If this happens you’ll most likely have a bunch of matplotlib errors as well – this error should be the last in the list. It can be fixed by starting FRELLED with the bizarre command :
LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libgfortran.so.3 /where/is/blender/blender FRELLED.blend
Where “where/is/blender/” is the location blender is installed. The directory “=/usr/lib/x86_64-linux-gnu/” may vary – you’ll have to find the location of libgfortran.so.3 yourself.
If this works, setup an alias so that Blender always starts with this command. If it doesn’t, don’t bother.
I get the error, “Command not found : os.system(‘python ExportFITS_standalone3D.py’)
Or something much like it anyway. If this happens on Windows (and probably Linux as well), you need to set your Python path. This is very easy – see http://docs.python.org/2/faq/windows.html for details. In brief, go to Control Panel -> System -> Advanced System Settings -> Environment Variables (see http://www.computerhope.com/issues/ch000549.htm). Find the “Path” system variable and hit “Edit”, then add the directory where Python was installed(e.g. C:\Python2.6\). Variables are separated with a semicolon.
Note : a number of sites which attempt to fix the "Blender can't find Python" problem claim you should set things like PYTHONPATH or PYTHONHOME. In my experience these actually cause problems and should be avoided. For me, it was setting the Path variable and only the Path variable that made things work.
While FRELLED has been designed to use as few external Python modules as possible when running in Blender (all of the operations on FITS files use external scripts, where there's usually no problem getting Python to work), it proved impossible to avoid using the os module.
What if I’m on a network ?
Take a nice skiing holiday, break something (an arm or a wrist preferably), and spend several months in hospital. That will prevent you from trying this terrible procedure. Currently, getting FRELLED to work on a Linux network is a lot less fun than stabbing yourself in the eye, or wrestling a mammoth. Don’t do it.
Bribing and/or seducing your network administrator are other non-recommended options which are currently untested. Alternatively, persuade them to install it locally on your own machine. Tell them you don't give a Suzanne if they'd rather network everything. Tough. It's really not worth the effort. And, while we're at it, if they tell you can't have a dedicated graphics card, get one anyway.
I want to edit FRELLED for my own nefarious purposes and have lots of different files to edit. I don’t want to keep changing the scripts every time I look at a different file – can I have changes propagate automatically from some master version ?
Yes you can ! At the moment, I haven’t decided if it’s better to keep the scripts internal or make them external, so you’ll have to do a little work to set this up if you need it.
Load FRELLED and exit the default “Display” menu with the “Q” key (with the mouse in the menu area). Use the “Text” menu to save the script wherever you want it. Then – and this is the important bit – save the Blender file. Now you can close down Blender, and go ahead and edit the script with whatever external editor you like best.
When you reload the Blender file, it will recognize if the external script differs from its internal version. A small question mark in a red box will be displayed :
Pressing this will give you three options to resolve the conflict : 1) Reload the file from disk - replacing the internal version (probably the one you want); 2) Make text internal – makes a separate, internal copy of the file, ignoring any external changes and breaking the link to the external version; 3) Ignore – ignores external changes, but doesn’t sever the link to the external version (so when you save it you still save over the external version).
Once the conflict is resolved, save the Blender file again. You’ll have to do this whenever you edit the script.
I wouldn’t recommend using both the internal Blender editor and another editor, as it can quickly become confusing as to which is the “real” version of the file. If you do, you should not only save the script when you make changes in Blender, but also the Blender file (otherwise it will still report a conflict).
If you’re editing multiple scripts, you’ll have to go through each one and decide if you want the internal or external version. You should do this before running the scripts, otherwise you’ll be running the internal versions whether you like it or not.
Can I load in different cubes into the same file ?
Yes you can ! This isn’t well-tested though, which is why it’s here and not in the main document. From the main screen, find the box next to the window selector that by default reads “SCE:3D”. Using the drop-down menu, press “ADD NEW”. Choose “Empty” and name make sure the name of the scene starts with the mode you want (you can type this in in the box, e.g. “3D2” – don’t make it a long name !!). You should then be able to load in a different cube in this scene (though more slowly than normal, since the scene doesn’t contain the pre-loaded blank objects onto which Blender displays the file and they have to be created) and flick back and forth to the other cube by changing scenes.
Does it matter which version of Python I have ?
Yes. If you have 32 bit Blender, you must have 32 bit Python. Or preferably get 64 bit versions of both.
You'll need to have a 2.6.X version of Python installed somewhere and Blender has to be able to find it. However, if you already have a different version of Python 2 installed with all the necessary modules and you don't want to install them again, that should be OK. Ensure that issuing the command "python" in a console starts that version of Python, but Blender can find Python 2.6. All of the FITS processing can be done via whatever version of Python 2 you like best, but Blender requires 2.6 to function properly.
I run Blender compiled with Python 2.6.2 but I have Blender find Python 2.6.5 or 2.6.8. So it shouldn't matter which particular version of 2.6 you get it to access.
Does it matter which versions of the Python modules I install ?
In principle no, not really - except for a minor problem with matplotlib v1.1.1. However, master beta tester Roberto Rodriguez provides the following foolproof guide to choosing the correct modules. He notes that there are sometimes problems with some module versions installing alongside others. The following should work on Windows :
Python 2.6 (I downloaded Windows x86 MSI Installer (2.6)): https://www.python.org/download/releases/2.6/ Numpy (I downloaded numpy-1.6.2.win32-py2.6.exe): https://pypi.python.org/pypi/numpy/1.6.2 Matplotlib (matplotlib-1.1.0.win32-py2.6.exe) : http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.1.0/ Pyfits (pyfits-3.0.13.win32-py2.6.exe): https://pypi.python.org/pypi/pyfits/3.0.13#downloads Added C:\Python26 to my Path environment variable (see above). Blender 2.49b (blender-2.49b-windows.exe) http://download.blender.org/release/Blender2.49b/ Choose default settings, especially when asked about the AppData directory FRELLED (v4.1 'Scarran' 2015 / 06 / 07) http://www.rhysy.net/frelled-archive.html Add .B file to C:\Users\YourUserName\AppData\Roaming\Blender Foundation\Blender\.blender
This has been confirmed to work even on a preview of Windows 10. Which is not bad considering that Blender 2.49 was released in 2009...
There may also be problems with matplotlib versions > 1.4, which uses the "six" library to support Python 3 in conjunction with Python 2. Apparently this can be extremely difficult to install on Windows 8. FRELLED was developed using matplotlib versions <1, so there's no loss of functionality by using earlier versions.
Do I have to install all these modules myself ? Can't I just grab a Python distribution with them all included ?
Yes you can ! The Anaconda distribution contains all the necessary modules, and more besides. This has been tested on Windows, Mac and Linux. On Windows, when you install Anaconda, be sure to choose the the option to install for all users, or it may not set the Python path correctly and Blender won't be able to find it.
Anaconda uses Python 2.7 by default. You'll need to use conda to install Python 2.6 :
conda update conda conda create -n py26 python=2.6 anaconda activate py26 (on Mac/Linux this becomes source activate py26)
Then set your paths to ...anaconda/envs/py26 (etc.)
Anaconda installs things a little differently so "import pyfits" will not work. You'll have to edit the scripts and replace this with "from astropy.io import fits as pyfits". This will be done automatically in the next version of FRELLED.
That's quite a lot of stuff about Python modules. I'm scared.
Yeah... but it isn't normally as bad as all that. For me, on Windows, all I had to do was add Python to my Path. On Linux I had to do absolutely nothing. Other people have experienced similarly effortless installations too. The above instructions are included for unusual cases.
The major unsolved problems are installing FRELLED on a Linux network and on a Mac. Well... some Macs.
I am having terrible problems with my Mac.
Well, it's a Mac. You should get rid of it before it eats your soul and kidnaps your goldfish. Macs are known to do that.
FRELLED is known to work on OS X 10.9.5, and probably on older versions too. After that the ghost of Steve Jobs, tormented by his failure to crush Bill Gates beneath his mighty feet, decided to break everything. So it's broken now, and doesn't work.
Macs : just say no !
Why doesn’t FRELLED have feature X ?
FRELLED was designed to do all the tasks I wished kvis could do. I don’t know what other users want, but if you ask nicely, I’ll probably add it if and when I can.
What’s with the stuff about the moose ?
Why FRELLED ?
Why not ? “Frelled” is a fictional word from the television show Farscape, which pre-empted the more popular “Frak” of Battlestar Galactica by several years. I don’t recall any episodes where the protagonists relied on having a good FITS viewer, but this is also true of ds9.
Notes for Developers
Here I'll explain in more detail how the code works and why it does things in ways that are perhaps not obvious - mostly to avoid various idiosyncrasies of Blender and optimise its display. I'll explain the layering structure, more about the difference between layers and scenes, and all kinds of wonderful things like that. Maybe even something about the code structure if I get really bored. For now, I'll just note that there's an archive of all previous FRELLED versions available here.
Features That Are Already On My "ToDo" List, So Don't Bother Asking For Them
Multi-wavelength data. Wouldn't that be super neat ?
WCSMask should include names. That's just obvious really.
Export should also produce QuickCat. Also obvious. It's just a good thing to do.
Make NED radius depend on beam size, or better yet FOV.
Better FOV for SDSS if using an interferometer.
Colour scheme for moment maps.
Moments other than 0, e.g. velocity maps.
Isosurfaces - there already exist Python scripts for Blender to do this, I'd just need to steal one.
If you've found a bug, want to request a new feature, or have any other FRELLED-based feedback, you can contact me here : email@example.com. If you want to be added to the mailing list to get notifications of new versions (a few times per year), let me know. If you're using FRELLED but don't particularly care to be added to the list, please please please let me know anyway ! It's good for my C.V. if I have an idea of how many people are really using FRELLED.
Alternatively, you can leave a message on the public FRELLED wiki page.
The "solidify" subroutine (used to transform the thin contour lines from matplotlib into thick Blender meshes) was written by Campbell Barton. The code is supplied with Blender under the usual license. The "filaments" subroutine was written by Richard Wunsch.
Extensive and intensive beta testing was performed by Roberto Rodriguez. Roberto is a master of all things installational and also discovered a galaxy that looks like the Loch Ness Monster. He found an uncounted number of bugs and solved a veritable plethora of installation problems, and generally makes the world a more functional place. "Princess" Olivia Keenan, on the other hand, just likes to test things to destruction. As a result the code is now far more robust to users doing really silly things and, hopefully, a lot easier to use than it was originally.
- ↑ Seriously. Even if reading manuals is against your religion.
- ↑ The moose had the magical ability to pronounce quotation marks.
- ↑ Princess Olivia was a warrior princess.
- ↑ Note that the velocity axis currently assumes the file is an HI data cube. The axes are robust to cubes spanning the entire sky or as small as a few minutes of time or arc, or a few km/s velocity. Data sets smaller than this, or without a WCS, will be given axes labelled by the dimensions of the cube in pixels.
- ↑ “Rendering” normally means applying various fancy lighting effects when generating the images. It is quite a bit slower, but a great deal more powerful, than simply taking screenshots of the current view.
Photos and videos are a great way to add visuals to your wiki. Find videos about your topic by exploring Wikia's Video Library.