Stack Exchange Network

Stack Exchange network consists of 181 Q&A communities including Stack Overflow , the largest, most trusted online community for developers to learn, share their knowledge, and build their careers. Visit Stack Exchange

Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.

Sign up to join this community

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

I have data on a dependent variable y and an explanatory one, x, and want to find out if there is a non-linear relationship between theses by running regressions where the data is divided in quartiles from the lowest to the highest value of x. So for example, what is the slope of x when x lies in the first quartile, say between 0 and 4 compared to when in the second, say between 4 and 10? I´m confused on how to accomplish this in Stata. Any help would be much appreciated!

Thanks in advance

Oscar

$\begingroup$

This is a highly problematic statistical practice. Besides greatly reducing power, it actually assumes a very strange relationship between X and Y. Suppose that X was linearly related to Y and that X did not have a uniform distribution. The the proposed approach would come up with a nonlinear relationship. It does not make sense for X to relate to Y by the shape of the marginal distribution of X. A better approach would be to fit a regression spline in the original X. You are also assuming that slope changes occur at certain percentiles of X which is hard to believe.

$\begingroup$ Frank, in your example with non-uniformly distributed X - if Oscar cuts X in subranges of equal width instead of cutting it at quartiles, will he still end up with a nonlinear relationship? $\endgroup$ – ttnphns Aug 21, 2011 at 7:19 $\begingroup$ Let me back up. The problem as originally stated can be dealt with using linear splines with intercepts restricted so that the segments meet at the join points (quartiles). Handouts at biostat.mc.vanderbilt.edu/rms show how [in the R rms package you can use the lsp function]. But there is no reason to assume that quartiles are where slope changes should happen, and cubic splines are likely to fit better, without having elbows in the fit. $\endgroup$ – Frank Harrell Aug 21, 2011 at 14:02 $\begingroup$ Why not use LOESS? Are there advantages of regression spline smoothing over LOESS? $\endgroup$ – ttnphns Aug 21, 2011 at 15:16 $\begingroup$ Loess would be an excellent choice here, much like regression splines. Both assume smoothness. Regression splines are easier to deal with to get confidence bands and hypothesis tests, and to generalize to the multivariable case. $\endgroup$ – Frank Harrell Aug 21, 2011 at 17:02 $\begingroup$ @Frank Harrell Thanks very much for the input! I'm trying out the b-spline function suggested above. What I actually have is data on tariffs as an explanatory variable and a proxy for customs evasion as the dependent variable. I wanted to know if there could be some fixed cost to evading customs so that at low tariffs evasion would be small (falt slope) and once tariffs get bigger one would see more of an effect. $\endgroup$ – Oscar Scheja Aug 23, 2011 at 14:24 $\begingroup$

Assuming you work with Stata 11 or above, so that you can easily use factor variables, you probably would want to do something like

    sysuse auto, clear
    xtile qprice = price , nq(4)
    reg mpg c.price##i.qprice
    testparm i(2/4).qprice#c.price i(2/4).qprice
If this solution is unclear, comment on it, I'll expand the explanation.
                $\begingroup$
                bsline ..., p(1) ...  is a nice shortcut:  See data.princeton.edu/eco572/smoothing2.html
                $\endgroup$
– Keith
                Aug 23, 2011 at 11:59
                $\begingroup$
                bspline, you mean. Yes, that certainly is an alternative. Frank Harrell mentions it in the other branch. I would personally stay with the cubic default though.
                $\endgroup$
– StasK
                Aug 23, 2011 at 13:54
        
Thanks for contributing an answer to Cross Validated!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.