Using pyPetal From an Input File

Using all modules of pyPetal may take 3 separate calls to different functions, in two different python environments. While this may be inconvenient (or a downright pain) to run normally (as seen in the full pyPetal tutorial), we have implemented tools to make this process more automated than the segmented form we have seen so far.

Making the Input File

pyPetal has functionality to run off of input using a TOML file, instead of inputting arguments through the python functions.

Here is an example of a pyPetal input file:

[inputs]
output_dir = 'toml_output/'
filenames = [ 'pypetal/pypetal/examples/dat/javelin_continuum.dat',
              '/home/stone28/pypetal/pypetal/examples/dat/javelin_yelm.dat',
              '/home/stone28/pypetal/pypetal/examples/dat/javelin_zing.dat' ]
line_names = ['continuum', 'yelm', 'zing']


[params]
    [params.general]
    plot = false
    verbose = true
    threads = 40
    lag_bounds = [-1000, 1000]
    file_fmt = 'ascii'

    [params.drw_rej]
    run = true
    use_for_javelin = true

    [params.detrend]
    run = true

    [params.pyccf]
    run = true
    nsim = 3500

    [params.pyzdcf]
    run = true
    nsim = 1000
    run_plike = true
    plike_dir = 'pypetal/plike_v4/'

    [params.pyroa]
    run = true
    nburn = 7000
    nchain = 10000
    add_var = true
    delay_dist = true
    subtract_mean = true
    psi_types = 'TruncGaussian'

    [params.javelin]
    run = true
    nburn = 50
    nchain = 50
    nwalker = 100
    lagtobaseline = 0.3

    [params.weighting]
    run = true
    k = 3
    rel_height = 0.5

We also could have contructed this file from utility functions in pyPetal, given the arguments:

[1]:

main_dir = 'pypetal/examples/dat/javelin_'
line_names = ['continuum', 'yelm', 'zing']
filenames = [ main_dir + x + '.dat' for x in line_names ]

output_dir = 'toml_output/'

[2]:

general_params = {
    'plot': False,
    'verbose': True,
    'threads': 40,
    'lag_bounds': [-1000,1000],
    'file_fmt': 'ascii'
}

run_drw_rej = True
drw_rej_params = {
    'use_for_javelin': True
}

run_detrend = False
detrend_params = {}

run_pyccf = True
pyccf_params = {
    'nsim': 3500
}

run_pyzdcf = True
pyzdcf_params = {
    'nsim': 1000,
    'run_plike': True,
    'plike_dir': 'pypetal/plike_v4/'
}

run_pyroa = True
pyroa_params = {
    'nburn': 7000,
    'nchain': 10000,
    'add_var': True,
    'delay_dist': True,
    'subtract_mean': True,
    'psi_types': 'TruncGaussian'
}

run_javelin = True
javelin_params = {
    'nburn': 50,
    'nchain': 50,
    'nwalker': 100,
    'lagtobaseline': 0.3
}

run_weighting = True
weighting_params = {
    'k': 3,
    'rel_height': 0.5
}


run_array = [run_drw_rej, run_detrend, run_pyccf, run_pyzdcf, run_pyroa,
             run_javelin, run_weighting]
params_array = [general_params,
                drw_rej_params, detrend_params, pyccf_params, pyzdcf_params, pyroa_params,
                javelin_params, weighting_params]

[3]:

from pypetal.fromfile.run_toml import make_toml

toml_dict = make_toml(output_dir, filenames,
                      run_array, params_array,
                      line_names=line_names,
                      filename='input.toml')

Running from the File

pyPetal

[4]:

from pypetal.fromfile.run_toml import run_from_toml1
_ = run_from_toml1('input.toml')


Performing DRW rejection
------------------------
jitter: True
nsig: 3
nwalker: 100
nburn: 300
nchain: 1000
clip: array
reject_data: [ True False False]
use_for_javelin: True
------------------------


Running pyCCF
-----------------
lag_bounds: [[-1000, 1000], [-1000, 1000]]
interp: 2.0000000001
nsim: 3500
mcmode: 0
sigmode: 0.2
thres: 0.8
nbin: 50
-----------------

Failed centroids:  0
Failed peaks:  0
Failed centroids:  0
Failed peaks:  0

Running pyZDCF
----------------------
nsim: 1000
minpts: 0
uniform_sampling: False
omit_zero_lags: True
sparse: auto
prefix: zdcf
run_plike: True
plike_dir: pypetal/plike_v4/
----------------------

Executing PLIKE
Executing PLIKE

Running PyROA
----------------
nburn: 7000
nchain: 10000
init_tau: [10.0, 10.0]
subtract_mean: True
div_mean: False
add_var: True
delay_dist: True
psi_types: ['TruncGaussian', 'TruncGaussian']
together: True
objname: pyroa
----------------

Initial Parameter Values
     A0            B0    σ0       A1           B1    τ1    Δ1    σ1      A2          B2    τ2    Δ2    σ2    Δ
-------  ------------  ----  -------  -----------  ----  ----  ----  ------  ----------  ----  ----  ----  ---
2.26171  -6.36307e-16  0.01  1.19302  4.11909e-16    10     1  0.01  0.5882  3.6042e-16    10     1  0.01   10
NWalkers=32

100%|██████████| 10000/10000 [22:24<00:00,  7.44it/s]

Filter: continuum
Mean Delay, error: 0.00 (fixed)
Filter: yelm
Mean Delay, error:  100.82627  (+   1.11104 -   1.07745)
Filter: zing
Mean Delay, error:  251.27615  (+   0.90225 -   0.83783)


Best Fit Parameters
     A0           B0        σ0       A1         B1       τ1        Δ1        σ1        A2          B2       τ2        Δ2         σ2        Δ
-------  -----------  --------  -------  ---------  -------  --------  --------  --------  ----------  -------  --------  ---------  -------
2.23246  -0.00374287  0.318231  1.17476  -0.021073  100.826  0.338319  0.186738  0.581659  0.00868483  251.276  0.344999  0.0855047  11.9584

pyPetal-jav

[1]:

from pypetal_jav.run_toml import run_from_toml_jav
_ = run_from_toml_jav('input.toml')


Running JAVELIN
--------------------
rm_type: spec
lagtobaseline: 0.3
laglimit: [[-1000, 1000], [-1000, 1000]]
fixed: True
p_fix: True
subtract_mean: True
nwalker: 100
nburn: 50
nchain: 50
output_chains: True
output_burn: True
output_logp: True
nbin: 50
metric: med
together: False
--------------------

run parallel chains of number 40
start burn-in
no priors on sigma and tau
penalize lags longer than 0.30 of the baseline
nburn: 50 nwalkers: 100 --> number of burn-in iterations: 5000
burn-in finished
save burn-in chains to /home/stone28/pypetal/toml_output/yelm/javelin/burn_rmap.txt
start sampling
sampling finished
acceptance fractions are
0.28 0.02 0.14 0.06 0.02 0.24 0.02 0.32 0.08 0.10 0.02 0.14 0.00 0.00 0.18 0.22 0.22 0.06 0.26 0.12 0.14 0.04 0.08 0.28 0.26 0.04 0.16 0.02 0.06 0.04 0.18 0.06 0.28 0.02 0.00 0.22 0.06 0.12 0.08 0.04 0.24 0.16 0.26 0.08 0.26 0.30 0.14 0.06 0.12 0.10 0.32 0.22 0.32 0.08 0.02 0.12 0.28 0.34 0.04 0.14 0.06 0.04 0.24 0.24 0.04 0.18 0.02 0.02 0.06 0.00 0.02 0.16 0.08 0.12 0.04 0.22 0.06 0.06 0.24 0.12 0.06 0.12 0.26 0.26 0.02 0.22 0.06 0.14 0.20 0.20 0.28 0.16 0.04 0.22 0.04 0.04 0.08 0.12 0.16 0.12
save MCMC chains to /home/stone28/pypetal/toml_output/yelm/javelin/chain_rmap.txt
save logp of MCMC chains to /home/stone28/pypetal/toml_output/yelm/javelin/logp_rmap.txt
HPD of sigma
low:    1.676 med    1.676 hig    1.676
HPD of tau
low:  263.428 med  263.428 hig  263.428
HPD of lag_yelm
low:   93.040 med  180.863 hig  540.571
HPD of wid_yelm
low:    0.228 med    0.594 hig    0.934
HPD of scale_yelm
low:    0.554 med    0.750 hig    0.914
covariance matrix calculated
covariance matrix decomposed and updated by U
run parallel chains of number 40
start burn-in
no priors on sigma and tau
penalize lags longer than 0.30 of the baseline
nburn: 50 nwalkers: 100 --> number of burn-in iterations: 5000
burn-in finished
save burn-in chains to /home/stone28/pypetal/toml_output/zing/javelin/burn_rmap.txt
start sampling
sampling finished
acceptance fractions are
0.08 0.06 0.10 0.04 0.06 0.10 0.02 0.04 0.04 0.08 0.02 0.16 0.06 0.06 0.08 0.00 0.10 0.12 0.08 0.06 0.20 0.12 0.02 0.04 0.08 0.00 0.10 0.04 0.26 0.02 0.04 0.10 0.08 0.16 0.20 0.02 0.06 0.12 0.06 0.10 0.08 0.14 0.00 0.12 0.02 0.02 0.26 0.04 0.08 0.10 0.18 0.02 0.08 0.10 0.12 0.12 0.08 0.04 0.02 0.04 0.06 0.08 0.18 0.06 0.14 0.00 0.12 0.06 0.02 0.00 0.14 0.00 0.04 0.06 0.12 0.16 0.00 0.16 0.02 0.00 0.08 0.06 0.16 0.04 0.02 0.02 0.12 0.18 0.16 0.00 0.04 0.12 0.08 0.00 0.04 0.16 0.06 0.06 0.00 0.04
save MCMC chains to /home/stone28/pypetal/toml_output/zing/javelin/chain_rmap.txt
save logp of MCMC chains to /home/stone28/pypetal/toml_output/zing/javelin/logp_rmap.txt
HPD of sigma
low:    1.676 med    1.676 hig    1.676
HPD of tau
low:  263.428 med  263.428 hig  263.428
HPD of lag_zing
low: -181.342 med  197.428 hig  550.095
HPD of wid_zing
low:    0.304 med    0.493 hig    0.770
HPD of scale_zing
low:    0.482 med    0.621 hig    0.762
covariance matrix calculated
covariance matrix decomposed and updated by U

Weighting

[1]:

from pypetal.fromfile.run_toml import run_from_toml2
_ = run_from_toml2('input.toml')

[3]:

from pypetal.utils import load

res, summary = load('toml_output/', verbose=True)


Prior pyPetal run
---------------------
DRW Rejection: True
pyCCF: True
pyZDCF: True
PyROA: True
JAVELIN: True
Weighting: True
---------------------

[4]:

res.keys()

[4]:

dict_keys(['drw_rej_res', 'pyccf_res', 'pyzdcf_res', 'plike_res', 'pyroa_res', 'javelin_res', 'weighting_res'])

[5]:

summary[0].keys()

[5]:

odict_keys(['k', 'n0_pyccf', 'peak_bounds_pyccf', 'peak_pyccf', 'lag_pyccf', 'lag_err_pyccf', 'frac_rejected_pyccf', 'n0_javelin', 'peak_bounds_javelin', 'peak_javelin', 'lag_javelin', 'lag_err_javelin', 'frac_rejected_javelin', 'n0_pyroa', 'peak_bounds_pyroa', 'peak_pyroa', 'lag_pyroa', 'lag_err_pyroa', 'frac_rejected_pyroa', 'rmax_pyccf', 'rmax_javelin', 'rmax_pyroa'])