You are not a real astronomer if you are not intimidated by the SDSS database.
The main issue is figuring out what to search for: this note is about finding the parameters you want to look for in the database, and passing them to the SQL search engine.
The list of all possible parameters and their keyword names within the SDSS catalogue can be found in the Schema Browser.
The SDSS catalogue is organized in tables, each of them containing different parameters.
Some are master tables which include others — for example the PhotoObjAll table encompasses all the photometry parameters.
By expanding the “TABLES” drop-down menu one can access all the tables, and from there the list of parameters within each table.
In case you look for a specific parameter, you might need to use the search function at the top of the page — which surprisingly doesn’t look up for all the keywords within the website (as in 99% of the internet sites!).
▸ Do the actual search
In SDSS there are plenty of ways to perform an object search.
In case you have a coordinate list, the best way to go is arguably to use the SQL CrossID (confusingly renamed “Object Crossid” throughout the website).
All you need to do is to specify the table you want to use and the parameters you want to retrieve (which you have from the procedure above).
The SQL engine will perform a proximity search (which you can tune), but in — the highly improbable — case you have at hand the run, rerun, camCol, field, and obj parameters of your object, you can perform a direct search.
The ideal would of course to search by SDSS ID, but I didn’t figure out yet how to do it in a convenient and simple way (NOTE: keep in mind that the SDSS IDs have changed since after DR7 because of a new run of object identification). If you know, please share in the comments under this post.
Tips on how to construct SQL searches are provided by the SDSS help page here.
Often times it is useful to exclude the [noisy] pixels at the edges of a FITS image.
With the word “edge” or “border”, I here refer to those pixels separating the area of the image with valid data pixels (value > 0), and that which simply fill the rest the canvas (value = 0, sometimes “-nan”, which I will call “NULL pixels”).
For example, SExtractor can be confused by the edges and return unphysically large sources there.
Look how much of a difference it makes when running SExtractor with the exact same setup, without masking (left), and masking (right) the image borders (NOTE: the masked border is represented by an horizontal dash in the left image):
Figure 1 – Left: SExtractor apertures obtained without masking the image borders. Right: SExtractor apertures obtained with the exact setup, but this time masking the image borders. Notice how the residual background (the background calculated by SExtractor is subtracted in these images) changes significantly also far away from the image borders.
PERL script to automatically mask borders
To mask the image edges I wrote a simple PERL script which automatically detects the image borders (as long as the NULL pixels have value = 0).
The border is “shrunk” (to exclude the [noisy] data pixels at the edge) and then masked.
▸ The code is available here for DOWNLOAD*:
The script is computationally intensive, so be patient! If you are not, use the “-verbose” keyword to check the progress.
*NOTE: it requires ds9, IRAF and some PERL packages installed (instructions within).
▸ Masking the borders of the image above
▸ Expanding SExtractor detections (segmentation map)
A very useful application is to “expand” (instead of “shrunking”) each of the sources detected by SExtractor. Since the “tail” of the emission of a source extends well beyond any radius (whether parametric or non-parametric) this method can be used to generate a conservative mask for all the objects in the image. Such mask is useful e.g. when examining the background.
In this case, we can use the SExtractor “segmentation mask” as a starting point. The segmentation mask indicates the contiguous pixels composing each object. The script will draw a contour around each detection, then expand and mask it.
NOTE: the file produced in this way must be inverted (i.e. switch 1 and 0) to obtain a standard mask.
▸ Funny stuff
The script seems to be pretty flexible, so you can draw contours of anything you like!
Yes, I should still tell you how to load the image of an octopus into a FITS file, but that’s for an other time …
Encircled Energy (EE)
60μm ▸ 60″
Positional Accuracy of PSC
From the Faint Source Catalogue (FSC):
▸Table I.B.1 – pg. I-12
@60μm (1-σ error) ▸ 4″x15″ (scan x cross-scan) – |b| > 10 [deg]
▸Table IV.A.2 – pg. IV-7
@60μm (1-σ error) ▸ 6.7″x13.4″ (scan x cross-scan) – F60μm > 1.9 [Jy]
@60μm (1-σ error) ▸ 7.4″x19.7″ (scan x cross-scan) – F60μm < 1.9 [Jy]
I had lots of troubles filing my tax reports for the income I got from my appointment at the CfA last year, so I decided to write some notes down in order to help myself (and any reader) next time.
The most common answer when asking information about american taxes is “seek a tax professional“, and this even comes from the employees of the tax offices.
Let me report here some useful information that cost me a lot of effort to figure out.
Information desk: 1-800-8291040
▸ The CfA withholds some Federal taxes (but no State taxes) based on the declarations that we provided in the application for the CfA appointment. The amount of withheld taxes is reported in the form 1042-S (provided by the employer). Basically, the expected expenses for food, housing, and transportations are subtracted from the “Gross Income” (which includes both salary and travel refunds) to give a “Net Income” (see boxes 2 and 4 in form 1042-S). The “Net Income” is the amount whose 14% is withheld as taxes (boxes 7 in form 1042-S).
To me, this value is meaningless because the calculations performed to get it cannot be reproduced in any way in neither the Federal Tax form (e.g. 1040NR-EZ) nor in the Massachusetts Tax form (e.g. NR-PY), so that the amount of taxes which is left to be payed/refunded is completely arbitrary – take this into account when filling the papers for the CfA appointment. I could get no explanation from the relevant personnel of CfA.
Massachusetts State Taxes
Information desk: 617-887-6367 (then press 1 for personal tax reports)
▸ The poverty limit is 8000$$: if the total amount of money earned in Massachusetts within the year is less than that, tax reports do not need to be filed.
▸ The web form does not work outside Massachusetts.
▸ Payment. From abroad, it is impossible to perform a direct payment when fling the tax form. In case any taxes are to be payed, the Tax office will send a bill to the foreign address.
Instructions for DOLPHOT
- The ACS Nearby Galaxy Survey Treasury – The author of DOLPHOT (i.e. Andrew Dolphin) is a co-author here. In Section 6 is given an overview on how DOLPHOT works. Gives separate advices for the ACS and WFPC2 cameras.
- Sarah Bruzzese’s thesis – She exchanged several mails with Andrew Dolphin and reports several common mistakes/misinterpretations.
- The posts in my BLOG.
Rejecting fake detections in crowded fields
▸ The DOLPHOT ACS manual suggests using Force1 =1 (i.e., all objects are assumed to be of type 1 == stars) in case of crowded fields.
When using this options, the manual suggests to use the following output parameters to discriminate real sources:
- χ (PSF fit Chi) – This is not a reduced χ. The degrees of freedom are:
FitSky =< 2 ⇒ 1 degree of freedom (only the height of the PSF is fitted)
FitSky = 3 ⇒ 2 degrees of freedom (the sky is fit as well)
FitSky = 4 ⇒ 2 degrees of freedom (?not explained in the manual?)
.. so calculate the reduced χ accordingly.
- sharpness (sharpness = 0 ⇒ perfect PSF; -0.3 < sharpness < 0.3 ⇒ good star) .
▸ An other suggestion for crowded fields (see The ACS Nearby Galaxy Survey Treasury, Section 6, 8th paragraph) is to try FitSky = 2.
▸ Cosmic Rays (CRs) cleaned images
In very few words, MultiDrizzle does the following:
1. Creates static masks (splits each chip)
2. Creates the WCS projected single science images (one for each file)
3. Creates a median image using the single science images
4. Creates the blotted (i.e. projects back to the chip frame) median image (splits each chip)
5. Compares the blotted median image against the original files to find and reject CRs (splits each chip)
6. Creates the final drizzled image
The MultiDrizzle manual suggests to compare the *_corr and *_crmask files against the original input in order to check the quality of rejection.
Here is the list of data products that can be inspected to evaluate the CR rejection:
- *_blt ▸ median image blotted back to the chip frame of reference
- *_cor.fits ▸ CR corrected image (for each chip) produced from the original input using the CR mask
- *_crmask ▸ CR mask generated comparing the blotted median images (*_blt) against the original input
- *_single_sci ▸ CR are still visible here since these images are produced at a previous step (step 2)
- *_single_wht ▸ weight image created from the comiantion of the DQ array (included in one of the extensions of the HST files, and possibly updated by MultiDrizzle itself – if workinplace = ‘yes’) and of the CR masks (?)
▸ driz_sep_bits – driz_final_bits
These parameters determines which is the maximum value of the sum of the flags for a given pixel to be considered.
Here I want to address to this issue: which is the best value to set the data quality flag when producing a drizzled image?
This subject is addressed in section 5.5.7 of the Multidrizzle Handbook (v3.0 – 2009).
The pixel flags for the HST ACS camera are reported in the ACS Data Handbook (Table 3.4 in manual version 6.0 – March 2011):
|1||Reed-Solomon decoding error|
|2||data replaced by fill value|
|4||bad detector pixel or vignetted pixel|
|8||masked by aperture feature|
|16||hot pixel (dark current > 0.08 e/sec)|
|32||CTE tail of hot pixel|
|64||warm pixel (dark current 0.02 to 0.08 e/sec)|
|128||bias structure (mostly bad columns)|
|256||saturation (full well or A-to-D)|
|512||bad pixel in reference file|
|1024||weak charge trap|
|4096||cosmic ray rejected by multidrizzle (based on all flt.fits files)|
|8192||cosmic ray rejected by acsrej|
Setting a driz_sep_bits (driz_final_bits) will have the effect to reject all of the pixels whose flags or sum of flags exceed (>, not >=) the value we set.
For example, if we set driz_sep_bits (driz_final_bits) = 8, we will reject pixels with flag 16 or more, but also those flagged both by a “bad detector pixel or vignetted pixel” and by a “masked by aperture feature” (4 + 8 =12 which exceeds the value we set, i.e. 8).
What happens in case a given pixel is flagged in all the fields that combine to create the drizzled image? In that case the value is determined by the parameter driz_sep_fillval (final_fillval; see MultiDrizzle help):
- driz_sep_fillval (final_fillval) = ‘INDEF’ ▸ the pixel is given a value as if the input had not been masked
- driz_sep_fillval (final_fillval) = constant ▸ the pixel is given constant value (e.g. -9.9)
The bottom line is that a “right” value for the parameters does not exist. It rather depends on the purpose. If the purpose is photometry, then consider that the source in the green circle must be rejected, since there is no way to recover its flux properly. If there was a third FLT image in the dither patter where the pixels were not rejected (in the case of the figure, the rejected pixels seem to be related to a charge transfer failure), it would be meaningful to keep those pixels (setting driz_sep_fillval (final_fillval) = ‘INDEF’ would exclude the input from the other two fields) – although that would lead to poor statistics.
In the case of the figure, it doesn’t really matter which of the two DRZ to keep. The only advantage of using the left one is to have a smoother image which will not confuse photometry software and will produce less spurious detections.
▸ Read Noise
In order to calculate the wight maps properly, MultiDrizzle needs the readout noise value to be reported in the header. The problem with ACS is that it has 2 amplifiers per CCD, for a total of 4 amplifiers, each with a different readout noise. In the FLT header, these values are recorded as:
READNSEA A READNSEA B READNSEA C READNSEA D
Which of these is chosen by MultiDrizzle (it uses only 1 value)?
To be more precise:
- When plotting their location over the reference image, the sources appeared separated in two spatially separated blocks, even if the blocks were correctly “tilted” according to the CCD distortion.
- Each block reproduced the relative position of the sources, but was shifted. At first, I thought this shift was equal to the one that the user must input in DOLPHOT (which is actually a first guess) in order to register the FLT files with a chosen reference (which, in my case, was the DRZ file downloaded from MAST along with the FLT files).
Curiously, the shift were instead:
▸ DX = (dx2 – dx1) / 2
▸ DY = (dy2 – dy1) / 2
where dx1 (dy1) was the x (y) shift of chip2 (WFC1) with respect to the reference image. I didn’t test it for chip1 (WFC2).
- The output coordinates list seemed to refer to only ONE of the WFC chips and indeed, in the dolphot output, I got results only for chip1 – column#2. I assume the output refers to the reference image, which is considered as 1 chip (since is a DRZ, and not an FLT image).
- As a reference image, I used a DRZ image produced by the ACS reduction pipeline (i.e., the one downloaded from MAST archive). This image has the same orientation of the FLT images, modulo the distortion. I know this generates issues with the sky estimate (the ACS pipeline subtracts the sky), but is certainly not the responsible for the coordinate shifts.
- I run ACSMASK, CALCSKY and SPLITGROUPS on each FLT image and on the reference.
- I skipped the ACSFITSDISTORT step, but I estimated the shift of each chip assuming that the center of each ACS WFC chip is located at (x,y)=(2048.5,1024.5) in the reference coordinate system.
- I run DOLPHOT2.0 (25 Feb 2012) — using the ACS PSFs from the DOLPHOT website.
- I prepared the region files using the DOLPHOT output.
Nimg = 4 #number of images (int) img0_file = reference.chip1 img1_file = NGC7793-POS1_1_B_flt.chip1 img2_file = NGC7793-POS1_1_B_flt.chip2 img3_file = NGC7793-POS1_2_B_flt.chip1 img4_file = NGC7793-POS1_2_B_flt.chip2 # img0_shift = 0 0 img1_shift = 57.8288465000001 1028.286212 img2_shift = -27.5632099999998 -1049.684777 img3_shift = 59.9029651000001 1030.34044 img4_shift = -25.4846766999999 -1047.61723 # img0_apsky = 15 25 img0_xform = 1 0 0 img1_apsky = 15 25 img1_xform = 1 0 0 img2_apsky = 15 25 img2_xform = 1 0 0 img3_apsky = 15 25 img3_xform = 1 0 0 img4_apsky = 15 25 img4_xform = 1 0 0 # RAper = 4 #photometry apeture size (flt) PSFPhot = 1 #photometry type (int/0=aper,1=psf,2=wtd-psf) FitSky = 3 #fit sky? (int/0=no,1=yes,2=small,3=with-phot) RSky0 = 15 #inner sky radius (flt>=RAper+0.5) Rsky1 = 35 #outer sky radius (flt>=RSky0+1) SkipSky = 2 #spacing for sky measurement (int>0) SkySig = 2.25 #sigma clipping for sky (flt>=1) xytfile = #position file for warmstart (str) xytpsf = #reference PSF for image subtraction SecondPass = 1 #second pass finding stars (int 0=no,1=yes) SigFind = 3.0 #sigma detection threshold (flt) SigFindMult = 0.85 #Multiple for quick-and-dirty photometry (flt>0) SigFinal = 3.0 #sigma output threshold (flt) MaxIT = 25 #maximum iterations (int>0) NoiseMult = 0.10 #noise multiple in imgadd (flt) FSat = 0.999 #fraction of saturate limit (flt) ApCor = 1 #find/make aperture corrections? (int 0=no,1=yes) Force1 = 1 #force type 1/2 (stars)? (int 0=no,1=yes) Align = 2 #align images? (int 0=no,1=const,2=lin,3=cube) Rotate = 1 #allow cross terms in alignment? (int 0=no, 1=yes) RCentroid = 2 #centroid box size (int>0) PosStep = 0.25 #search step for position iterations (flt) dPosMax = 2.5 #maximum single-step in position iterations (flt) RCombine = 1.5 #minimum separation for two stars for cleaning (flt) RPSF = 10 #PSF size (int>0) SigPSF = 5.0 #min S/N for psf parameter fits (flt) PSFres = 1 #make PSF residual image? (int 0=no,1=yes) psfstars = #Coordinates of PSF stars psfoff = 0.0 #coordinate offset (PSF system - dolphot system) # UseWCS = 0 # DEFAULT Notice that the UseWCS function was off. This parameter allows to choose whether to use the WCS from the reference image in order to perform an initial guess for the alignment of the FLT fields to the reference itself (the real shifts are calculated through tweakshifts ?).
After trying several solutions without any success, I decided to contact the author of DOLPHOT, i.e., Andrew Dolphin. Following his suggestions, I managed to get to the following conclusions:
▸ UseWCS = 1
▸ all input guesses for the shifts between the FLT fields and the reference to 0
solved the coordinates issue.
Probably the coordinates shift I was using were incorrect. In fact, going back to my log files, I realized that no match was happening between the sources found in the reference and those in the FLTs. I assume that the error is a combination of two effects:
▸ a wrong guess for the shifts – the pipeline I was using had specific assumptions on the location of the center of the CCDs (chips) with respect to the reference image. These were probably true only if the canvas of the drizzled image is the smallest size that can accommodate the image (this can achieved leaving blank the final_uotnx and final_outnx parameters of MultiDrizzle).
▸ a small tolerance radius when performing the coordinates match – but this is an in-built feature of DOLPHOT and I ignore how to change it.
- Using as reference image a drizzled FLT (that I produced according to the method described later), instead of the MAST DRZ, dramatically increased the number of stars actually used for for alignment (“matched”). This number was roughly 1/3 to 2/3 of the “stars for alignment” — which is what is expected considering that the FLT completely overlap the reference image. The RMS scatter was ~ 0.2 pixels. I assume that my drizzled image produced a better result with respect to what happened to the pipelined DRZ image due to a better cosmic ray rejection.
Creating a reference image
The reference image is used to calculate the WCS coordinates of the sources detected in the FLT images.
By construction, the FLT files are the un-distorded images (in the sense that no distortion correction has been applied) as they appear on the ACS CCDs. DOLPHOT is able to apply the distortions, shifts and scalings, and can even deal with rotations. Therefore – in principle – an FLT image can be aligned to any appropriate reference image, whatever its scale and rotation is. I tested that the alignment did work when using a reference image aligned to WCS. Though, the sources coordinates identified by DOLPHOT in this way presented a residual offset (whose amount varied across the image) with respect to the actual locations. A rigid shift of approximately (dx , dy) = (-0.5 pixel , -0.5 pixel) is present also when using a non-WCS aligned reference: this is related to the discrepancy between DOLPHOT and IRAF pixel center definition (DOLPHOT ACS manual, Section 3.5).
In practice, is much easier if the reference image is just a copy of the FLT image which accounts for the distortion effects (so that to achieve the alignment no rotations or scalings must be applied – but only a rigid shift). Such image can be generated using Pyraf MultiDrizzle.
Here is the MultiDrizzle configuration:
iraf.multidrizzle.input = 'filename_flt.fits' iraf.multidrizzle.output = '' iraf.multidrizzle.ra = '' iraf.multidrizzle.dec = '' iraf.multidrizzle.mdriztab = 'no' iraf.multidrizzle.refimage = '' iraf.multidrizzle.runfile = '' iraf.multidrizzle.workinplace = 'no' iraf.multidrizzle.coeffs = 'header' iraf.multidrizzle.context = 'yes' iraf.multidrizzle.clean = 'no' iraf.multidrizzle.group = '' iraf.multidrizzle.build = 'yes' iraf.multidrizzle.shiftfile = '' iraf.multidrizzle.staticfile = '' # iraf.multidrizzle.static = 'yes' iraf.multidrizzle.static_sig = 4.0 # iraf.multidrizzle.skysub = 'no' iraf.multidrizzle.skywidth = 0.1 iraf.multidrizzle.skystat = 'median' iraf.multidrizzle.skylower = 'INDEF' iraf.multidrizzle.skyupper = 'INDEF' iraf.multidrizzle.skyclip = 5 iraf.multidrizzle.skylsigma = 4.0 iraf.multidrizzle.skyusigma = 4.0 iraf.multidrizzle.skyuser = '' # iraf.multidrizzle.driz_separate = 'yes' iraf.multidrizzle.driz_sep_outnx = '' iraf.multidrizzle.driz_sep_outny = '' iraf.multidrizzle.driz_sep_kernel = 'turbo' iraf.multidrizzle.driz_sep_wt_scl = 'exptime' iraf.multidrizzle.driz_sep_scale = 0.05 iraf.multidrizzle.driz_sep_pixfrac = 1.0 iraf.multidrizzle.driz_sep_rot = 'INDEF' iraf.multidrizzle.driz_sep_fillval = '-9.9' #iraf.multidrizzle.driz_sep_fillval = 'INDEF' iraf.multidrizzle.driz_sep_bits = 4192 # iraf.multidrizzle.median = 'yes' iraf.multidrizzle.median_newmasks = 'yes' iraf.multidrizzle.combine_type = 'median' iraf.multidrizzle.combine_nlow = 0 iraf.multidrizzle.combine_nhigh = 1 iraf.multidrizzle.combine_lthresh = '-8.8' #iraf.multidrizzle.combine_lthresh = '0.000001' iraf.multidrizzle.combine_hthresh = 'INDEF' iraf.multidrizzle.combine_grow = 1 # iraf.multidrizzle.blot = 'yes' iraf.multidrizzle.blot_interp = 'poly5' iraf.multidrizzle.blot_sinscl = 1.0 # iraf.multidrizzle.driz_cr = 'yes' iraf.multidrizzle.driz_cr_corr = 'no' iraf.multidrizzle.driz_cr_snr = '4.5 3.0' iraf.multidrizzle.driz_cr_grow = 1 iraf.multidrizzle.driz_cr_ctegrow = 0 iraf.multidrizzle.driz_cr_scale = '1.2 0.7' # iraf.multidrizzle.driz_combine = 'yes' iraf.multidrizzle.final_wht_type = 'EXP' iraf.multidrizzle.final_outnx = '' iraf.multidrizzle.final_outny = '' iraf.multidrizzle.final_kernel = 'square' iraf.multidrizzle.final_wt_scl = 'exptime' iraf.multidrizzle.final_scale = 0.05 iraf.multidrizzle.final_pixfrac = 1.0 iraf.multidrizzle.final_rot = 'INDEF' iraf.multidrizzle.final_fillval = 'INDEF' iraf.multidrizzle.final_bits = 0 # iraf.multidrizzle.mode = 'al'
- final_rot = ‘INDEF’ – this guarantees that reference image will not be rotated (only distortion map will be applied).
- build = ‘yes’ – this parameter states that the output image will be a multi-extension file containing the science, weight mask, and the contex images produced by MultiDrizzle. This is fundamental for DOLPHOT to read the file correctly (notice that the single chip files that must be fed to DOLPHOT have a somehow similar format).
- skysub = ‘no’ – according to what is described here (Sec 3.2.1), this is required to obtain a correct photometry (although I am still not sure how photometry is performed on the reference image).
Notes on the output
- The coordinates of the sources are relative to the reference frame.
- The output sources have approximately a (dx , dy) = (-0.5 pixel , -0.5 pixel) rigid shift with respect to the reference image. This is related to the discrepancy between DOLPHOT and IRAF pixel center definition (DOLPHOT ACS manual, Section 3.5). In case the reference image is WCS aligned, or has different scale with respect to the FLTs images, the correction is much more complex since it involves rotation and scaling (personally, I didn’t try to do it).
- The reference image is only used for aligning the other images. No photometry is done on it.
- The distortion maps used by MultiDrizzle are not necessarily the same as the ones used by DOLPHOT. Therefore, at the step in which the FLT chips are aligned to the reference image, DOLPHOT has to apply a distortion that may result different from that of the reference image (created by MultiDrizzle). This should not produce any issue on the locations of the sources anyway, since:
▸ the difference between different versions of distortion maps are little
▸ the ouput coordinates are calculated based only on the WCS of the reference image.
Even elliptical galaxies – which are expected to be relaxed systems characterized by an isotropic distribution of stellar orbits – do not always show a completely smooth distribution of their starlight. On the contrary, it is observed that about ~40% of isolated early-type galaxies show evidence for sharp discontinuities in their surface brightness.
The genesis of such remnant structures has been long identified in numerical simulations of galaxy interactions (e.g. Toomre & Toomre, 1972, ApJ, 178, 623; Quinn, 1984, ApJ, 279, 596; Hernquist & Quinn, 1989, ApJ, 342, 1).
Here I found an interesting animation on how shells form.
Following, a list of fancy animations of galaxy formation/interaction.
▸ Animation of shell formation:
▸ Interactive java galaxy merger simulator (scientific explanation at http://burro.cwru.edu/JavaLab/GalCrashWeb/ellipt.html):
> Proposal preparation checklist:
Policies, Procedures & Phase I Proposal Instructions, Table 7.2
> APT webpage:
> Phase I proposal roadmap:
> Scientific Justification PDF templates: