Can i include the product of two random variables? Or do I risk collinearity?





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}







1












$begingroup$


I have a model in which I want to predict Y.



My regressors X, are x1 and x2.



For some reason I believe that it would also be useful to include into the model:




  • x3 = x1 * x2

  • x4 = x1 / x2


Can I use a regressors x1, x2, x3 and x4 altogether or do I risk perfect collinearity problem.
I know for instance that using x5 = x1 + x2 would yield perfect collinearity and hence a completely useless regressor.






regression linear-model multicollinearity







share|cite|improve this question









$endgroup$








  • 2




    $begingroup$
    x3 is commonly called an interaction term.
    $endgroup$
    – COOLSerdash
    6 hours ago


















1












$begingroup$


I have a model in which I want to predict Y.



My regressors X, are x1 and x2.



For some reason I believe that it would also be useful to include into the model:




  • x3 = x1 * x2

  • x4 = x1 / x2


Can I use a regressors x1, x2, x3 and x4 altogether or do I risk perfect collinearity problem.
I know for instance that using x5 = x1 + x2 would yield perfect collinearity and hence a completely useless regressor.






regression linear-model multicollinearity







share|cite|improve this question









$endgroup$








  • 2




    $begingroup$
    x3 is commonly called an interaction term.
    $endgroup$
    – COOLSerdash
    6 hours ago














1












1








1





$begingroup$


I have a model in which I want to predict Y.



My regressors X, are x1 and x2.



For some reason I believe that it would also be useful to include into the model:




  • x3 = x1 * x2

  • x4 = x1 / x2


Can I use a regressors x1, x2, x3 and x4 altogether or do I risk perfect collinearity problem.
I know for instance that using x5 = x1 + x2 would yield perfect collinearity and hence a completely useless regressor.






regression linear-model multicollinearity







share|cite|improve this question









$endgroup$




I have a model in which I want to predict Y.



My regressors X, are x1 and x2.



For some reason I believe that it would also be useful to include into the model:




  • x3 = x1 * x2

  • x4 = x1 / x2


Can I use a regressors x1, x2, x3 and x4 altogether or do I risk perfect collinearity problem.
I know for instance that using x5 = x1 + x2 would yield perfect collinearity and hence a completely useless regressor.






regression linear-model multicollinearity




regression linear-model multicollinearity






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked 6 hours ago









scugn1zz0scugn1zz0

84




84








  • 2




    $begingroup$
    x3 is commonly called an interaction term.
    $endgroup$
    – COOLSerdash
    6 hours ago














  • 2




    $begingroup$
    x3 is commonly called an interaction term.
    $endgroup$
    – COOLSerdash
    6 hours ago








2




2




$begingroup$
x3 is commonly called an interaction term.
$endgroup$
– COOLSerdash
6 hours ago




$begingroup$
x3 is commonly called an interaction term.
$endgroup$
– COOLSerdash
6 hours ago










2 Answers
2






active

oldest

votes


















0












$begingroup$

No, you don’t risk collinearity because $x_i$ are not linearly dependent in general, i.e. the below equation has just one solution holding for all possible $x_i$:
$$a_1x_1+a_2x_2+a_3x_3+a_4x_4=0$$
And that is $a_i=0$.



In $x_5=x_1+x_2$ case, the following equation has non-zero solutions such that $a_1=a_2=-a_5$: $$a_1x_1+a_2x_2+a_5x_5=0$$






share|cite|improve this answer











$endgroup$





















    2












    $begingroup$

    While you won't have perfect collinearity (as per your question), you do risk multicollinearity issues with your two additional regressors.



    While they're not algebraically linear combinations of the two predictors, it can be the case that these variables (x1-x4) in a particular sample might lay close to to a linear subspace - with the typical consequences of near-multicollinearity.



    For example, if the two original variates both have very small coefficients of variation then their product can be quite closely related to their sum (or some other linear combination if they're dissimilar in size). This can happen even if the original variables are not highly correlated.



    Similarly, high correlation can happen with the ratio and the difference.






    share|cite|improve this answer











    $endgroup$









    • 1




      $begingroup$
      +1. As I understand things, you want to center your variables (subtract the mean), which will help to reduce correlation between x1, x2, and x1 * x2.
      $endgroup$
      – Wayne
      1 hour ago










    • $begingroup$
      It would certainly help; but then we can come back to other examples; in the case of certain kinds of relationships between x1 and x2, the collection of x1,x2,x1*x2 and x1/x2 can still result in near-multicollinearity
      $endgroup$
      – Glen_b♦
      59 mins ago












    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "65"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f401487%2fcan-i-include-the-product-of-two-random-variables-or-do-i-risk-collinearity%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0












    $begingroup$

    No, you don’t risk collinearity because $x_i$ are not linearly dependent in general, i.e. the below equation has just one solution holding for all possible $x_i$:
    $$a_1x_1+a_2x_2+a_3x_3+a_4x_4=0$$
    And that is $a_i=0$.



    In $x_5=x_1+x_2$ case, the following equation has non-zero solutions such that $a_1=a_2=-a_5$: $$a_1x_1+a_2x_2+a_5x_5=0$$






    share|cite|improve this answer











    $endgroup$


















      0












      $begingroup$

      No, you don’t risk collinearity because $x_i$ are not linearly dependent in general, i.e. the below equation has just one solution holding for all possible $x_i$:
      $$a_1x_1+a_2x_2+a_3x_3+a_4x_4=0$$
      And that is $a_i=0$.



      In $x_5=x_1+x_2$ case, the following equation has non-zero solutions such that $a_1=a_2=-a_5$: $$a_1x_1+a_2x_2+a_5x_5=0$$






      share|cite|improve this answer











      $endgroup$
















        0












        0








        0





        $begingroup$

        No, you don’t risk collinearity because $x_i$ are not linearly dependent in general, i.e. the below equation has just one solution holding for all possible $x_i$:
        $$a_1x_1+a_2x_2+a_3x_3+a_4x_4=0$$
        And that is $a_i=0$.



        In $x_5=x_1+x_2$ case, the following equation has non-zero solutions such that $a_1=a_2=-a_5$: $$a_1x_1+a_2x_2+a_5x_5=0$$






        share|cite|improve this answer











        $endgroup$



        No, you don’t risk collinearity because $x_i$ are not linearly dependent in general, i.e. the below equation has just one solution holding for all possible $x_i$:
        $$a_1x_1+a_2x_2+a_3x_3+a_4x_4=0$$
        And that is $a_i=0$.



        In $x_5=x_1+x_2$ case, the following equation has non-zero solutions such that $a_1=a_2=-a_5$: $$a_1x_1+a_2x_2+a_5x_5=0$$







        share|cite|improve this answer














        share|cite|improve this answer



        share|cite|improve this answer








        edited 6 hours ago

























        answered 6 hours ago









        gunesgunes

        7,0751215




        7,0751215

























            2












            $begingroup$

            While you won't have perfect collinearity (as per your question), you do risk multicollinearity issues with your two additional regressors.



            While they're not algebraically linear combinations of the two predictors, it can be the case that these variables (x1-x4) in a particular sample might lay close to to a linear subspace - with the typical consequences of near-multicollinearity.



            For example, if the two original variates both have very small coefficients of variation then their product can be quite closely related to their sum (or some other linear combination if they're dissimilar in size). This can happen even if the original variables are not highly correlated.



            Similarly, high correlation can happen with the ratio and the difference.






            share|cite|improve this answer











            $endgroup$









            • 1




              $begingroup$
              +1. As I understand things, you want to center your variables (subtract the mean), which will help to reduce correlation between x1, x2, and x1 * x2.
              $endgroup$
              – Wayne
              1 hour ago










            • $begingroup$
              It would certainly help; but then we can come back to other examples; in the case of certain kinds of relationships between x1 and x2, the collection of x1,x2,x1*x2 and x1/x2 can still result in near-multicollinearity
              $endgroup$
              – Glen_b♦
              59 mins ago
















            2












            $begingroup$

            While you won't have perfect collinearity (as per your question), you do risk multicollinearity issues with your two additional regressors.



            While they're not algebraically linear combinations of the two predictors, it can be the case that these variables (x1-x4) in a particular sample might lay close to to a linear subspace - with the typical consequences of near-multicollinearity.



            For example, if the two original variates both have very small coefficients of variation then their product can be quite closely related to their sum (or some other linear combination if they're dissimilar in size). This can happen even if the original variables are not highly correlated.



            Similarly, high correlation can happen with the ratio and the difference.






            share|cite|improve this answer











            $endgroup$









            • 1




              $begingroup$
              +1. As I understand things, you want to center your variables (subtract the mean), which will help to reduce correlation between x1, x2, and x1 * x2.
              $endgroup$
              – Wayne
              1 hour ago










            • $begingroup$
              It would certainly help; but then we can come back to other examples; in the case of certain kinds of relationships between x1 and x2, the collection of x1,x2,x1*x2 and x1/x2 can still result in near-multicollinearity
              $endgroup$
              – Glen_b♦
              59 mins ago














            2












            2








            2





            $begingroup$

            While you won't have perfect collinearity (as per your question), you do risk multicollinearity issues with your two additional regressors.



            While they're not algebraically linear combinations of the two predictors, it can be the case that these variables (x1-x4) in a particular sample might lay close to to a linear subspace - with the typical consequences of near-multicollinearity.



            For example, if the two original variates both have very small coefficients of variation then their product can be quite closely related to their sum (or some other linear combination if they're dissimilar in size). This can happen even if the original variables are not highly correlated.



            Similarly, high correlation can happen with the ratio and the difference.






            share|cite|improve this answer











            $endgroup$



            While you won't have perfect collinearity (as per your question), you do risk multicollinearity issues with your two additional regressors.



            While they're not algebraically linear combinations of the two predictors, it can be the case that these variables (x1-x4) in a particular sample might lay close to to a linear subspace - with the typical consequences of near-multicollinearity.



            For example, if the two original variates both have very small coefficients of variation then their product can be quite closely related to their sum (or some other linear combination if they're dissimilar in size). This can happen even if the original variables are not highly correlated.



            Similarly, high correlation can happen with the ratio and the difference.







            share|cite|improve this answer














            share|cite|improve this answer



            share|cite|improve this answer








            edited 3 hours ago

























            answered 4 hours ago









            Glen_b♦Glen_b

            215k23417770




            215k23417770








            • 1




              $begingroup$
              +1. As I understand things, you want to center your variables (subtract the mean), which will help to reduce correlation between x1, x2, and x1 * x2.
              $endgroup$
              – Wayne
              1 hour ago










            • $begingroup$
              It would certainly help; but then we can come back to other examples; in the case of certain kinds of relationships between x1 and x2, the collection of x1,x2,x1*x2 and x1/x2 can still result in near-multicollinearity
              $endgroup$
              – Glen_b♦
              59 mins ago














            • 1




              $begingroup$
              +1. As I understand things, you want to center your variables (subtract the mean), which will help to reduce correlation between x1, x2, and x1 * x2.
              $endgroup$
              – Wayne
              1 hour ago










            • $begingroup$
              It would certainly help; but then we can come back to other examples; in the case of certain kinds of relationships between x1 and x2, the collection of x1,x2,x1*x2 and x1/x2 can still result in near-multicollinearity
              $endgroup$
              – Glen_b♦
              59 mins ago








            1




            1




            $begingroup$
            +1. As I understand things, you want to center your variables (subtract the mean), which will help to reduce correlation between x1, x2, and x1 * x2.
            $endgroup$
            – Wayne
            1 hour ago




            $begingroup$
            +1. As I understand things, you want to center your variables (subtract the mean), which will help to reduce correlation between x1, x2, and x1 * x2.
            $endgroup$
            – Wayne
            1 hour ago












            $begingroup$
            It would certainly help; but then we can come back to other examples; in the case of certain kinds of relationships between x1 and x2, the collection of x1,x2,x1*x2 and x1/x2 can still result in near-multicollinearity
            $endgroup$
            – Glen_b♦
            59 mins ago




            $begingroup$
            It would certainly help; but then we can come back to other examples; in the case of certain kinds of relationships between x1 and x2, the collection of x1,x2,x1*x2 and x1/x2 can still result in near-multicollinearity
            $endgroup$
            – Glen_b♦
            59 mins ago


















            draft saved

            draft discarded




















































            Thanks for contributing an answer to Cross Validated!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f401487%2fcan-i-include-the-product-of-two-random-variables-or-do-i-risk-collinearity%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            宮崎県

            Image
            みやざきけん 宮崎県 高千穂峡 宮崎県旗 宮崎県徽章 国 日本 地方 九州地方 団体コード 45000-6 ISO 3166-2:JP JP-45 面積 7,735.32 km 2 （境界未定部分あり） 総人口 1,079,727 人 （推計人口、2018年10月1日） 人口密度 140 人/km 2 隣接都道府県 大分県、熊本県、鹿児島県 県の木 フェニックス（1966年9月3日制定） ヤマザクラ（2003年2月6日制定） オビスギ（飫肥杉）（2003年2月6日制定） 県の花 ハマユウ（1964年12月22日制定） 県の鳥 コシジロヤマドリ（1964年12月22日制定） 他のシンボル 県の歌：宮崎県民歌（1964年制定） マスコット：みやざき犬（2011年11月11日制定） 宮崎県庁 知事 河野俊嗣 法人番号 4000020450006 所在地 〒 880-8501 宮崎県宮崎市橘通東2丁目10番1号 北緯31度54分39.6秒東経131度25分25.9秒 座標: 北緯31度54分39.6秒 東経131度25分25.9秒 外部リンク 宮崎県 ■ ― 市 / ■ ― 町 / ■ ― 村 地理院地図 Google Bing GeoHack MapFan Mapion Yahoo! NAVITIME ゼンリン ウィキポータル 日本の都道府県/宮崎県   表示   ウィキプロジェクト 宮崎県 （みやざきけん）は、九州地方南東部に位置する日本の県である。県庁所在地は宮崎市。 県木である「フェニックス」に代表される南国情緒豊かな気候から、1960年代には日南地区を中心とした新婚旅行のメッカだった。現在も春季のプロ野球などのキャンプ地として知られる。 目次 1 地理・地域 1.1 地形 1.2 自然公園 1.3 気候 1.4 自治体 1.5 合併済みの市町村 2 歴史 2.1 神話 2...
            Read more

            濃尾地震

            Image
            濃尾地震 『岐阜市街大地震之図』 歌川国利画 本震 発生日 1891年（明治24年）10月28日 発生時刻 6時38分50秒（JST） 震央 日本 岐阜県本巣郡西根尾村（現・本巣市） 北緯35度35分 東経136度20分（ 地図 ） 規模    マグニチュード（M）8.0 最大震度    震度7： 注1 福井県今立郡鯖江町、愛知県葉栗郡大田島村、東春日井郡勝川村 地震の種類 直下型地震 被害 死傷者数 死者7,273人、負傷者17,175人 注1：当時の震度階級では「激烈」 プロジェクト:地球科学 プロジェクト:災害 テンプレートを表示 濃尾地震 （のうびじしん）は、1891年（明治24年）10月28日に濃尾地方で発生した、日本史上最大の内陸地殻内地震。「 美濃・尾張地震 （みの・おわりじしん）」とも呼ばれている。辛卯の年に発生したことから 辛卯震災 と呼んでいる報告書もある。 目次 1 概要 1.1 震源断層 2 被害 3 各地の震度 4 前兆現象 5 報道 6 学術的な意義 7 地震防災 8 脚注 9 関連項目 10 外部リンク 概要 濃尾地震発生当時の根尾谷断層 濃尾地震を引き起こした根尾谷断層 写真中央を斜めに走る段差が根尾谷断層 濃尾地震は、1891年10月28日6時38分50秒に発生した。震源は、岐阜県本巣郡西根尾村（現・本巣市）、北緯35度35分、東経136度20分付近。河角廣 (1951) は岐阜市付近（北緯35.6度、東経136.6度）に震央を仮定し規模 M K = 7.0 を与え [1] 、マグニチュードは M = 8.4 に換算されているが、明治・大正期の地震については0.5程度大きく見積もられているとされる [2] 。また、震央距離と震度との関係など当時のデータから後にM8.0 [3] とも推定される。アメリカ地質調査所 (USGS) でも8.0としている [4] 。「根尾谷断層帯」が活動した典型的な内陸地殻内地震（いわゆる直下型地震）であり、これは記録が残っている日本の内陸...
            Read more

            シテ島

            Image
            シテ島 Île de la Cité シテ島（トゥルネル橋からの眺望） 所在地 フランス･パリ 座標 北緯48度51分17秒 東経2度20分45秒 座標: 北緯48度51分17秒 東経2度20分45秒 面積 0.225 km² 最高標高 48.854722 m  プロジェクト 地形 テンプレートを表示 シテ島 （シテとう、仏 : Île de la Cité）は、サン＝ルイ島と並ぶ、パリの中心部を流れるセーヌ川の中州である。パリ市内で最も古い歴史をもつパリ1区及び4区に属し、“パリ発祥の地”とも称される。ユリウス・カエサルの『ガリア戦記』では、すでに紀元前1世紀にパリシイ族 (Parisii) が住んでいたことが述べられている。 目次 1 概要 2 歴史ギャラリー 3 名所 4 シテ島の交通 4.1 橋 4.2 鉄道 5 ギャラリー 6 関連項目 7 脚注 8 外部リンク 概要 東半分が4区に、西半分は1区に属する。 ノートルダム大聖堂とパリ警視庁とパリ市立病院（ オテル・デュー・ド・パリ （フランス語版） 、直訳では「パリの神の館」）がある。病院の名前は、旅の病人を宿泊させ、病になれば看病した教会、巡礼教会からの発展の名残である。近代外科医学の開拓者で、床屋医者の出身でもある外科医アンブロワーズ・パレが勤務していた病院としてもその名が知られている。位置関係は大聖堂が警視庁庁舎と向き合い、警視庁の隣に市立病院。 また、この島はとても狭いので、ここの住民たちは夜も一定の時間を過ぎるとトイレの水は流さない等、互いに気配りしつつ一緒に住んでいくことの不便さを味わってきた。ここから「シテ島に住む者」を表す古仏語：citeain （現代仏語：シトワイヤン citoyen 、現代英語：シティズン citizen）の意味が転じ、「 市民 」という言葉が成立したという（仏語：cité は英語：city に相当する）。 中世以来からの狭く入り組んだ路地は、19世紀に入りセーヌ川左岸と共に貧民窟化していた。そうした狭く不衛生で入り組んだ空間は、2月革命に見るようなバリケード戦といった民衆騒...
            Read more