Add remittances to OG-Core #971

rickecon · 2024-08-21T14:30:34Z

This adds remittances to OG-Core and does some heavy updates to the documentation. This PR:

Updates all of the documentation.
- Adds remittances to all instances of the household budget constraint
- Rewrites bequests and transfers components of household budget constraint in terms of individual variables in all instances
- Adds a household transfers section to households.md with subsections on bequests, remittances, government transfers, and universal basic income
- Changes all instances of $p_t Y_t$ to $Y_t
- Updates the steady-state equilibrium algorithm description in equilibrium.md
- Added updates to the government pensions descriptions in government.md and added pensions to all instances of the household budget constraint.
- Updates the docstrings in tax.py for the wealth tax ETR and MTR functions. The code is right. I just thought there was a clearer specification of the equations in LaTeX.
Adds remittances to the OG-Core code
- Adds aggregate remittances function get_RM() to aggregates.py
- Adds household remittances function get_rm() to household.py
- Adds four new remittance parameters: alpha_RM_1, g_RM, alpha_RM_T, eta_RM
- We model aggregate remittances as a percent of GDP in the first period, then growing at a specified rate that can deviate from the country growth rate until the cutoff rule period, after which the remittance growth rate trends back to the population growth rate. We also model remittances in reforms as being a percentage of baseline GDP. In this way, if remittance parameters are not changed in the reform, remittances remain at their baseline levels. The only way they change is if their parameter values are changed.
- Adds 3 tests using the test_get_RM() function in test_aggregates.py
- Adds 4 tests using the test_get_rm() function in test_household.py
- Changes the initial_guess_r_SS in two tests in test_SS.py because they were not solving with their current values
Increases RC_SS steady-state resource constraint tolerance from 1e-9 to 1e-8. because on of the tests in test_run_TPI.py::test_run_TPI_full_run[Baseline, small open] was failing due to `RuntimeError: Steady state aggregate resource constraint not satisfied. The maximum absolute resource constraint error was 2.29575914e-09.
Increases RC_TPI transition path resource constraint tolerance from 1e-5 to 1e-4 in because one test_run_TPI_full_run() test was failing in test_TPI.py with a resource constraint error just bigger than 1e-5 (1.4459913381864586e-05 for [Baseline, M=3 non-zero Kg]).
Updated three directory path references that were out of date in test_run_example.py.
Updated expected value tuples and dictionaries in test_txfunc.py.

cc: @jdebacker

docs/book/content/theory/equilibrium.md

codecov-commenter · 2024-08-26T08:54:24Z

Codecov Report

Attention: Patch coverage is 83.82353% with 11 lines in your changes missing coverage. Please review.

Project coverage is 70.14%. Comparing base (92f7594) to head (1f14820).
Report is 7 commits behind head on master.

Files with missing lines	Patch %	Lines
ogcore/TPI.py	47.61%	11 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #971      +/-   ##
==========================================
+ Coverage   69.98%   70.14%   +0.16%     
==========================================
  Files          20       20              
  Lines        4977     5038      +61     
==========================================
+ Hits         3483     3534      +51     
- Misses       1494     1504      +10

Flag	Coverage Δ
unittests	`70.14% <83.82%> (+0.16%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
ogcore/SS.py	`72.48% <100.00%> (+0.50%)`	⬆️
ogcore/__init__.py	`100.00% <100.00%> (ø)`
ogcore/aggregates.py	`100.00% <100.00%> (ø)`
ogcore/constants.py	`100.00% <ø> (ø)`
ogcore/household.py	`87.87% <100.00%> (+0.60%)`	⬆️
ogcore/parameter_plots.py	`78.34% <100.00%> (+0.05%)`	⬆️
ogcore/parameters.py	`83.12% <100.00%> (+0.21%)`	⬆️
ogcore/tax.py	`100.00% <ø> (ø)`
ogcore/TPI.py	`36.45% <47.61%> (+0.62%)`	⬆️

... and 1 file with indirect coverage changes

rickecon · 2024-09-13T08:21:08Z

@jdebacker. All the GH Action CI tests have now passed. Some of the local tests in test_TPI.py are still failing.

I have two of the 11 tests of test_run_TPI_full_run() failing because of RuntimeError: Transition path equlibrium not found (RC_error). I still need to look into why this is happening. This one might be a little trickier.
I have 9 other errors (2 in test_run_TPI() and 7 in test_run_TPI_extra()) in which I am getting the error ValueError: operands could not be broadcast together with shapes (80,7) (40,2). This will be easy to fix, as it is just either an expected tuple or test tuple that is missing remittances.

I have successfully run the run_ogcore_example.py script on my local machine:

Baseline SS equilibrium

SS debt =  0.625296240150551 0.007949921472992331
IO:  (1, 1) , C:  (1,)
Foreign debt holdings =  0.2501184960602204
Foreign capital holdings =  0.05620836287422404
resource constraint:  [-4.07937573e-14]
Checking constraints on capital, labor, and consumption.
	There were no violations of the constraints on labor  supply.
	There were no violations of the constraints on  consumption.
Maximum error in labor FOC =  1.0791367799356522e-13
Maximum error in savings FOC =  7.216449660063518e-14
JUST SAVED SS output to  /Users/richardevans/Docs/Economics/OSE/OG-Core/run_examples/OG-Core-Example/OUTPUT_BASELINE/SS/SS_vars.pkl

Baseline TP equilibrium (35 min, 47 sec)

Iteration: 21
	Distance: 7.501911757006711e-06
Max absolute value resource constraint error: 8.459743900193772e-07
Checking time path for violations of constraints.
Max Euler error, savings:  3.4733882436910335e-12
Max Euler error labor supply:  1.3300471835009375e-12
Time path iteration complete.

Reform SS equilibrium

SS debt =  0.6228816994331038 0.00791922336885464
IO:  (1, 1) , C:  (1,)
Foreign debt holdings =  0.24915267977324151
Foreign capital holdings =  0.05260193171249955
resource constraint:  [-2.34014197e-14]
Checking constraints on capital, labor, and consumption.
	There were no violations of the constraints on labor  supply.
	There were no violations of the constraints on  consumption.
Maximum error in labor FOC =  1.1102230246251565e-13
Maximum error in savings FOC =  8.615330671091215e-14
JUST SAVED SS output to  /Users/richardevans/Docs/Economics/OSE/OG-Core/run_examples/OG-Core-Example/OUTPUT_REFORM/SS/SS_vars.pkl

Reform TP equilibrium (39 min, 29 sec)

Iteration: 21
	Distance: 6.942822337487318e-06
Max absolute value resource constraint error: 8.089560793039796e-07
Checking time path for violations of constraints.
Max Euler error, savings:  4.652778162750337e-12
Max Euler error labor supply:  1.1874945471390674e-12
Time path iteration complete.
It took 2368.6731803417206 seconds to get that part done.
run time =  2368.6732518672943
total time was  4526.702739238739
Percentage changes in aggregates: Year                    Variable  2021  2022  2023  2024  2025  2026  2027  2028  2029  2030  2021-2030    SS
0                    GDP ($Y_t$) -0.17 -0.14 -0.22 -0.22 -0.21 -0.19 -0.16 -0.12 -0.10 -0.07      -0.16 -0.39
1            Consumption ($C_t$) -0.08  1.05  0.13 -0.13 -0.24 -0.32 -0.37 -0.42 -0.43 -0.44      -0.12 -0.72
2          Capital Stock ($K_t$) -0.37 -0.41 -0.66 -0.73 -0.75 -0.74 -0.71 -0.64 -0.60 -0.54      -0.62 -1.51
3                  Labor ($L_t$) -0.05  0.00  0.02  0.05  0.08  0.11  0.14  0.16  0.17  0.18       0.08  0.22
4     Real interest rate ($r_t$) -3.05 -2.94 -2.64 -2.52 -2.47 -2.45 -2.45 -2.58 -2.61 -2.65      -2.64 -1.40
5                      Wage rate -0.11 -0.14 -0.24 -0.27 -0.29 -0.30 -0.30 -0.28 -0.27 -0.25      -0.25 -0.61

I also plotted the aggregate remittances time series from the last two cases tested in test_aggregates.py in the test_get_RM() function. The first case, listed below as "RM2" is the following parameterization in which all the other parameters are at the default_parameters.json values:

alpha_RM_1 = 0.05
alpha_RM_T = 0.05
g_RM = ((np.exp(g_y) * (1 + g_n_ss)) - 1) * np.ones(T + S)

In this case, aggregate remittances start and end at 5% of GDP. And they grow over the time series at the aggregate long-run growth rate of the economy. The plot dips below its initial level because the long-run aggregate growth rate is bigger than the actual growth rate of GDP in the early periods.

The second lin is labeled "RM3" and is represented by a similar parameterization in which the only difference is that the growth rate of remittances over the first $T_{G1}$ periods is a half a percent greater than the long-run aggregate growth rate:

g_RM = ((np.exp(g_y) * (1 + g_n_ss)) - 1 + 0.005) * np.ones(T + S)

The code for creating this plot is:

import numpy as np
from ogcore.parameters import Specifications
from ogcore import aggregates as aggr
import matplotlib.pyplot as plt

Y_RM1 = 0.625
p_RM2= Specifications(baseline=True)
p_RM2.alpha_RM_1 = 0.05
p_RM2.alpha_RM_T = 0.05
p_RM2.g_RM =(
    ((np.exp(p_RM2.g_y_annual) * (1 + p_RM2.g_n_ss)) - 1) *
    np.ones(p_RM2.T + p_RM2.S)
)

Y_RM2 = Y_RM1 * np.ones(p_RM2.T + p_RM2.S)
RM2 = aggr.get_RM(Y_RM2, p_RM2, "TPI")

p_RM3 = copy.deepcopy(p_RM2)
p_RM3.g_RM = (
   ((np.exp(p_RM2.g_y_annual) * (1 + p_RM2.g_n_ss)) - 1 + 0.005) *
   np.ones(p_RM2.T + p_RM2.S)
)
RM3 = aggr.get_RM(Y_RM2, p_RM3, "TPI")

plt.plot(np.arange(2025, 2025 + p_RM3.T), RM3[:p_RM3.T], label='RM3')
plt.plot(np.arange(2025, 2025 + p_RM3.T), RM2[:p_RM3.T], label='RM2')
plt.vlines(
    [2025 + p_RM3.tG1, 2025 + p_RM3.tG2], 0.0292, 0.0336, color='black',
    linestyle='--'
)
plt.xlabel(r"Year $t$")
plt.ylabel(r"Remittances $\hat{RM}_t$")
plt.legend()

rickecon · 2024-09-17T02:52:55Z

@jdebacker. I think this PR is now ready for review and to be merged if accepted. In addition to running the run_ogcore_example.py script successfully as documented above, I ran all the tests--both GH Actions and local. The results are below. All the tests pass except for the 9 detailed below. And my recommendation is that we solve those remaining 9 errors in separate PRs. In summary:

There is 1 test in test_TPI.py that is failing (test_run_TPI_full_run[Baseline, small open]). The TPI equilibrium converges, but the resource constraint error is too big (0.07786113397046332). This case probably needs to be looked at. I recommend we do this in another PR.
The 8 local tests in test_demographics.py fail, but the 8 GH Actions tests pass. All 8 failing tests fail with this error (OSError: pytest: reading from stdin while output is captured! Consider using -s.) and the output is all stalled asking for Please enter your UN API token (press return if you do not have one):. This is clearly an issue with the UN API token functionality and needs to be fixed. I recommend we do this in another PR.

(ogcore-dev) richardevans@Richards-MacBook-Pro-2 OG-Core % git status
On branch remit
nothing to commit, working tree clean
(ogcore-dev) richardevans@Richards-MacBook-Pro-2 OG-Core % pytest
============================= test session starts ==============================
platform darwin -- Python 3.12.5, pytest-8.3.2, pluggy-1.5.0
rootdir: /Users/richardevans/Docs/Economics/OSE/OG-Core
configfile: pytest.ini
testpaths: ./tests
plugins: cov-5.0.0, anyio-4.4.0, xdist-3.6.1
collected 580 items                                                            

tests/test_SS.py ...................................                     [  6%]
tests/test_TPI.py ...........F...............                            [ 10%]
tests/test_aggregates.py ........................................        [ 17%]
tests/test_basic.py ....                                                 [ 18%]
tests/test_demographics.py F...FFFF..F...FF                              [ 21%]
tests/test_elliptical_u_est.py .......                                   [ 22%]
tests/test_execute.py .                                                  [ 22%]
tests/test_firm.py ..................................................... [ 31%]
................                                                         [ 34%]
tests/test_fiscal.py ...................                                 [ 37%]
tests/test_household.py ................................................ [ 45%]
..                                                                       [ 46%]
tests/test_output_plots.py ............................................. [ 53%]
..                                                                       [ 54%]
tests/test_output_tables.py ..............                               [ 56%]
tests/test_parameter_plots.py ........................................   [ 63%]
tests/test_parameter_tables.py .......                                   [ 64%]
tests/test_parameters.py ..............                                  [ 67%]
tests/test_pensions.py ...............................                   [ 72%]
tests/test_run_example.py ..                                             [ 72%]
tests/test_run_ogcore.py .                                               [ 73%]
tests/test_tax.py .................................                      [ 78%]
tests/test_txfunc.py ...............................                     [ 84%]
tests/test_user_inputs.py .........                                      [ 85%]
tests/test_utils.py .................................................... [ 94%]
...............................                                          [100%]
=========================== short test summary info ============================
FAILED tests/test_TPI.py::test_run_TPI_full_run[Baseline, small open] - assert False
FAILED tests/test_demographics.py::test_get_pop_objs_read_UN_data - OSError: pytest: reading from stdin while output is captured!  Consider usi...
FAILED tests/test_demographics.py::test_imm_smooth - OSError: pytest: reading from stdin while output is captured!  Consider usi...
FAILED tests/test_demographics.py::test_get_fert - OSError: pytest: reading from stdin while output is captured!  Consider usi...
FAILED tests/test_demographics.py::test_get_mort - OSError: pytest: reading from stdin while output is captured!  Consider usi...
FAILED tests/test_demographics.py::test_infant_mort - OSError: pytest: reading from stdin while output is captured!  Consider usi...
FAILED tests/test_demographics.py::test_custom_series - OSError: pytest: reading from stdin while output is captured!  Consider usi...
FAILED tests/test_demographics.py::test_infer_pop_nones - OSError: pytest: reading from stdin while output is captured!  Consider usi...
FAILED tests/test_demographics.py::test_data_download - OSError: pytest: reading from stdin while output is captured!  Consider usi...
FAILED tests/test_run_example.py::test_run_ogcore_example - FileNotFoundError: [Errno 2] No such file or directory: '/Users/richardevan...
FAILED tests/test_run_example.py::test_run_ogcore_example_output - FileNotFoundError: [Errno 2] No such file or directory: '/Users/richardevan...
FAILED tests/test_txfunc.py::test_txfunc_est[DEP] - assert False
FAILED tests/test_txfunc.py::test_txfunc_est[GS] - assert False
FAILED tests/test_txfunc.py::test_tax_func_loop - assert False
FAILED tests/test_txfunc.py::test_tax_func_estimate - assert False
======== 9 failed, 571 passed, 16801 warnings in 44933.83s (12:28:53) =========

rickecon · 2024-09-18T20:08:22Z

@jdebacker. I made the following updates:

Updated all instances of the resource constraint function aggregates.get_resource_constraint() in the model.
Updated instances of the resource constraint to include "- RM_t" in the docstring for aggregate.get_resource_constraint(), in the unstationarized resource constraint equation and corresponding text in the market_clearing.md chapter, and in the stationarized resource constraint equation in the stationarized.md chapter.
Added my un_api_token.txt file to my OG-Core main directory. Now all the demographics.py local tests pass. I did have to update the test_infant_mort() test expected value from 0.00477758 to 0.00496921.

rickecon · 2024-09-19T21:26:28Z

@jdebacker. Now all the tests are passing, both GH Actions and local (see output below). This PR should now be ready to go.

(ogcore-dev) richardevans@Richards-MacBook-Pro-2 OG-Core % pytest
============================= test session starts ==============================
platform darwin -- Python 3.12.5, pytest-8.3.2, pluggy-1.5.0
rootdir: /Users/richardevans/Docs/Economics/OSE/OG-Core
configfile: pytest.ini
testpaths: ./tests
plugins: cov-5.0.0, anyio-4.4.0, xdist-3.6.1
collected 580 items

tests/test_SS.py ...................................                     [  6%]
tests/test_TPI.py ...........................                            [ 10%]
tests/test_aggregates.py ........................................        [ 17%]
tests/test_basic.py ....                                                 [ 18%]
tests/test_demographics.py ................                              [ 21%]
tests/test_elliptical_u_est.py .......                                   [ 22%]
tests/test_execute.py .                                                  [ 22%]
tests/test_firm.py ..................................................... [ 31%]
................                                                         [ 34%]
tests/test_fiscal.py ...................                                 [ 37%]
tests/test_household.py ................................................ [ 45%]
..                                                                       [ 46%]
tests/test_output_plots.py ............................................. [ 53%]
..                                                                       [ 54%]
tests/test_output_tables.py ..............                               [ 56%]
tests/test_parameter_plots.py ........................................   [ 63%]
tests/test_parameter_tables.py .......                                   [ 64%]
tests/test_parameters.py ..............                                  [ 67%]
tests/test_pensions.py ...............................                   [ 72%]
tests/test_run_example.py ..                                             [ 72%]
tests/test_run_ogcore.py .                                               [ 73%]
tests/test_tax.py .................................                      [ 78%]
tests/test_txfunc.py ...............................                     [ 84%]
tests/test_user_inputs.py .........                                      [ 85%]
tests/test_utils.py .................................................... [ 94%]
...............................                                          [100%]
============= 580 passed, 16801 warnings in 45150.36s (12:32:30) ==============

rickecon added 10 commits August 16, 2024 14:07

Updated documentation files

e16ec26

Updated equilibrium.md

e05f91d

Merge remote-tracking branch 'origin/master' into remit

1c007e4

Updated default_parameters.json

53fae74

Updated documentation

affd65e

Updated docstrings in tax.py

9f8654b

Updated government.md

4a8064a

Updated documentation

20869b3

Updated docstring in tax.py

25dc700

Updated documentation

7762317

rickecon marked this pull request as draft August 21, 2024 14:30

Updated SS algorithm in code and docs

4d24afb

rickecon mentioned this pull request Aug 22, 2024

Is GDP specified correctly in SS.py algorithm? #972

Open

rickecon linked an issue Aug 22, 2024 that may be closed by this pull request

Is GDP specified correctly in SS.py algorithm? #972

Open

Updated equilibrium.md

f611ead

jdebacker reviewed Aug 22, 2024

View reviewed changes

docs/book/content/theory/equilibrium.md Outdated Show resolved Hide resolved

rickecon added 4 commits August 24, 2024 05:03

Updated documentation for equilibrium step

f704600

Reverted Y update line in SS.py

0b420f0

Updated file paths in run_ogcore_example.py

4dc4868

Updated output paths in run_ogcore_example.py

6181fe3

rickecon removed a link to an issue Aug 24, 2024

Is GDP specified correctly in SS.py algorithm? #972

Open

Restore default_parameters.json to orig

969466b

rickecon and others added 4 commits August 29, 2024 12:04

Updated aggregates.py and households.md

4a9b7b6

fix pension subscripts

4d7b18c

put pensions in household budget constraint

801af44

define aggregate pension spending

cd80888

jdebacker mentioned this pull request Aug 29, 2024

Some fixes to pension system rickecon/OG-Core#2

Merged

rickecon added 2 commits September 3, 2024 12:20

Updated government.md and households.md

7efcce0

Merge branch 'remit' into pr-971

a7503a6

rickecon added 5 commits September 12, 2024 13:38

Black formatted TPI.py and parameters.py

1277c20

Updated tests_TPI.py

9ed1539

Updated test_TPI.py

d5b4762

Updated twist_doughnut data files

5f64455

Updated test_inner_loop in test_TPI.py

9130a0a

rickecon added 10 commits September 15, 2024 13:14

Updated test_TPI.py::test_run_TPI tests

cc92773

Update test_TPI.py

7f59b9b

Updated test_TPI.py

7ae3744

Updated CHANGELOG.md

bdb254f

UPDATED CHANGELOG.md

80bc551

Updated CHANGELOG.md and test_run_example.py

321c04a

Black formatted test_run_example.py

8341e5f

Updated test_run_example.py

a3ed07b

Updated CHANGELOG.md and tests

678997b

Black formatted test_txfunc.py

58a152d

Updated CHANGELOG.md

9cf79d1

rickecon mentioned this pull request Sep 18, 2024

[WIP] Parameterize adjustments to baseline spending levels in reform policy #987

Open

rickecon added 6 commits September 18, 2024 14:26

Update aggregate remittances in resource constraint in code

3714290

Black formatted TPI.py

ca46d5f

Updated aggregate remittances in resource constraint

abad20e

Update test_infant_mort() test in test_demographics.py

802bf2e

Restored RC_SS val to 1e-09

a96eb77

Update default_parameters.json values for RC_SS and RC_TPI

bc6db91

rickecon added 3 commits September 19, 2024 00:13

Updated files

3169b8f

Updated test_TPI.py and test_run_example.py

796a779

Black formatted test_run_example.py

1f14820

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add remittances to OG-Core #971

Add remittances to OG-Core #971

rickecon commented Aug 21, 2024 •

edited

Loading

codecov-commenter commented Aug 26, 2024 •

edited

Loading

rickecon commented Sep 13, 2024

rickecon commented Sep 17, 2024

rickecon commented Sep 18, 2024 •

edited

Loading

rickecon commented Sep 19, 2024

Add remittances to OG-Core #971

Are you sure you want to change the base?

Add remittances to OG-Core #971

Conversation

rickecon commented Aug 21, 2024 • edited Loading

codecov-commenter commented Aug 26, 2024 • edited Loading

Codecov Report

rickecon commented Sep 13, 2024

rickecon commented Sep 17, 2024

rickecon commented Sep 18, 2024 • edited Loading

rickecon commented Sep 19, 2024

rickecon commented Aug 21, 2024 •

edited

Loading

codecov-commenter commented Aug 26, 2024 •

edited

Loading

rickecon commented Sep 18, 2024 •

edited

Loading