Can the alpha, lambda values of a glmnet object output determine whether ridge or Lasso?












2












$begingroup$


Given a glmnet object using train() where trControl method is "cv" and number of iterations is 5, I obtained that the bestTune alpha and lambda values are alpha=0.1 and lambda= 0.007688342. On running the glmnet object, I notice that the alpha values start from 0.1.
Can the inference here be that the method used is Lasso and not ridge because of the non-negative alpha value?



In general, can the values of alpha, lambda indicate which model is being used?






regression generalized-linear-model cross-validation caret







share|cite|improve this question







New contributor




red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$

















    2












    $begingroup$


    Given a glmnet object using train() where trControl method is "cv" and number of iterations is 5, I obtained that the bestTune alpha and lambda values are alpha=0.1 and lambda= 0.007688342. On running the glmnet object, I notice that the alpha values start from 0.1.
    Can the inference here be that the method used is Lasso and not ridge because of the non-negative alpha value?



    In general, can the values of alpha, lambda indicate which model is being used?






    regression generalized-linear-model cross-validation caret







    share|cite|improve this question







    New contributor




    red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.







    $endgroup$















      2












      2








      2





      $begingroup$


      Given a glmnet object using train() where trControl method is "cv" and number of iterations is 5, I obtained that the bestTune alpha and lambda values are alpha=0.1 and lambda= 0.007688342. On running the glmnet object, I notice that the alpha values start from 0.1.
      Can the inference here be that the method used is Lasso and not ridge because of the non-negative alpha value?



      In general, can the values of alpha, lambda indicate which model is being used?






      regression generalized-linear-model cross-validation caret







      share|cite|improve this question







      New contributor




      red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.







      $endgroup$




      Given a glmnet object using train() where trControl method is "cv" and number of iterations is 5, I obtained that the bestTune alpha and lambda values are alpha=0.1 and lambda= 0.007688342. On running the glmnet object, I notice that the alpha values start from 0.1.
      Can the inference here be that the method used is Lasso and not ridge because of the non-negative alpha value?



      In general, can the values of alpha, lambda indicate which model is being used?






      regression generalized-linear-model cross-validation caret




      regression generalized-linear-model cross-validation caret






      share|cite|improve this question







      New contributor




      red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.











      share|cite|improve this question







      New contributor




      red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      share|cite|improve this question




      share|cite|improve this question






      New contributor




      red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.









      asked 6 hours ago









      red4life93red4life93

      161




      161




      New contributor




      red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.





      New contributor





      red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






      red4life93 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






















          2 Answers
          2






          active

          oldest

          votes


















          2












          $begingroup$

          As far as I understand glmnet, $alpha=0$ would actually be a ridge penalty, and $alpha=1$ would be a Lasso penalty (rather than the other way around) and as far as glmnet is concerned you can fit those end cases.



          The penalty with $alpha=0.1$ would be fairly similar to the ridge penalty but it is not the ridge penalty; if it's not considering $alpha$ below $0.1$ you can't necessarily infer much more than that just from the fact that you had that endpoint. If you know that an $alpha$ value that was only slightly larger was worse then it would be likely that a larger range might have chosen a smaller $alpha$, but it doesn't suggest it would have been $0$; I expect it would not. If the grid of values is coarse it may well have been that a larger value than $0.1$ would be better.



          [You may want to check whether there was some other reason that $alpha$ might have been at an endpoint; e.g. I seem to recall $lambda$ got set to an endpoint in forecasting if coefficients for lambdaOpt were not saved.]






          share|cite|improve this answer











          $endgroup$





















            1












            $begingroup$

            Absolutely! The $alpha$ parameter can be adjusted to either fit a Lasso or a Ridge regression (or something in between). Recall that the loss function which Elastic Net minimizes is $$frac{1}{2N}sum^N_{i=1}(y_i-beta_0-x_i^tbeta)^2+lambdasum_{j=1}^p(frac{1}{2}(1-alpha)beta_j^2+alpha|beta_j|).$$
            Focus on the second big sum (the one multiplied by $lambda$). If you let $alpha=1$, the first term inside this sum becomes $0$, and the whole function becomes exactly the function that Lasso minimizes (or the Lasso loss function). If you let $alpha=0$, the second term becomes $0$ and you are left with Ridge.



            You can check the loss for Ridge and Lasso in this book and for elastic net in this paper.






            share|cite|improve this answer









            $endgroup$













            • $begingroup$
              This looks like a good answer but can you edit to include citations for the hyperlinks? Over time, links die.
              $endgroup$
              – Sycorax
              6 hours ago











            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "65"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });






            red4life93 is a new contributor. Be nice, and check out our Code of Conduct.










            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396748%2fcan-the-alpha-lambda-values-of-a-glmnet-object-output-determine-whether-ridge-o%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            2 Answers
            2






            active

            oldest

            votes








            2 Answers
            2






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            2












            $begingroup$

            As far as I understand glmnet, $alpha=0$ would actually be a ridge penalty, and $alpha=1$ would be a Lasso penalty (rather than the other way around) and as far as glmnet is concerned you can fit those end cases.



            The penalty with $alpha=0.1$ would be fairly similar to the ridge penalty but it is not the ridge penalty; if it's not considering $alpha$ below $0.1$ you can't necessarily infer much more than that just from the fact that you had that endpoint. If you know that an $alpha$ value that was only slightly larger was worse then it would be likely that a larger range might have chosen a smaller $alpha$, but it doesn't suggest it would have been $0$; I expect it would not. If the grid of values is coarse it may well have been that a larger value than $0.1$ would be better.



            [You may want to check whether there was some other reason that $alpha$ might have been at an endpoint; e.g. I seem to recall $lambda$ got set to an endpoint in forecasting if coefficients for lambdaOpt were not saved.]






            share|cite|improve this answer











            $endgroup$


















              2












              $begingroup$

              As far as I understand glmnet, $alpha=0$ would actually be a ridge penalty, and $alpha=1$ would be a Lasso penalty (rather than the other way around) and as far as glmnet is concerned you can fit those end cases.



              The penalty with $alpha=0.1$ would be fairly similar to the ridge penalty but it is not the ridge penalty; if it's not considering $alpha$ below $0.1$ you can't necessarily infer much more than that just from the fact that you had that endpoint. If you know that an $alpha$ value that was only slightly larger was worse then it would be likely that a larger range might have chosen a smaller $alpha$, but it doesn't suggest it would have been $0$; I expect it would not. If the grid of values is coarse it may well have been that a larger value than $0.1$ would be better.



              [You may want to check whether there was some other reason that $alpha$ might have been at an endpoint; e.g. I seem to recall $lambda$ got set to an endpoint in forecasting if coefficients for lambdaOpt were not saved.]






              share|cite|improve this answer











              $endgroup$
















                2












                2








                2





                $begingroup$

                As far as I understand glmnet, $alpha=0$ would actually be a ridge penalty, and $alpha=1$ would be a Lasso penalty (rather than the other way around) and as far as glmnet is concerned you can fit those end cases.



                The penalty with $alpha=0.1$ would be fairly similar to the ridge penalty but it is not the ridge penalty; if it's not considering $alpha$ below $0.1$ you can't necessarily infer much more than that just from the fact that you had that endpoint. If you know that an $alpha$ value that was only slightly larger was worse then it would be likely that a larger range might have chosen a smaller $alpha$, but it doesn't suggest it would have been $0$; I expect it would not. If the grid of values is coarse it may well have been that a larger value than $0.1$ would be better.



                [You may want to check whether there was some other reason that $alpha$ might have been at an endpoint; e.g. I seem to recall $lambda$ got set to an endpoint in forecasting if coefficients for lambdaOpt were not saved.]






                share|cite|improve this answer











                $endgroup$



                As far as I understand glmnet, $alpha=0$ would actually be a ridge penalty, and $alpha=1$ would be a Lasso penalty (rather than the other way around) and as far as glmnet is concerned you can fit those end cases.



                The penalty with $alpha=0.1$ would be fairly similar to the ridge penalty but it is not the ridge penalty; if it's not considering $alpha$ below $0.1$ you can't necessarily infer much more than that just from the fact that you had that endpoint. If you know that an $alpha$ value that was only slightly larger was worse then it would be likely that a larger range might have chosen a smaller $alpha$, but it doesn't suggest it would have been $0$; I expect it would not. If the grid of values is coarse it may well have been that a larger value than $0.1$ would be better.



                [You may want to check whether there was some other reason that $alpha$ might have been at an endpoint; e.g. I seem to recall $lambda$ got set to an endpoint in forecasting if coefficients for lambdaOpt were not saved.]







                share|cite|improve this answer














                share|cite|improve this answer



                share|cite|improve this answer








                edited 6 hours ago

























                answered 6 hours ago









                Glen_b♦Glen_b

                213k22412762




                213k22412762

























                    1












                    $begingroup$

                    Absolutely! The $alpha$ parameter can be adjusted to either fit a Lasso or a Ridge regression (or something in between). Recall that the loss function which Elastic Net minimizes is $$frac{1}{2N}sum^N_{i=1}(y_i-beta_0-x_i^tbeta)^2+lambdasum_{j=1}^p(frac{1}{2}(1-alpha)beta_j^2+alpha|beta_j|).$$
                    Focus on the second big sum (the one multiplied by $lambda$). If you let $alpha=1$, the first term inside this sum becomes $0$, and the whole function becomes exactly the function that Lasso minimizes (or the Lasso loss function). If you let $alpha=0$, the second term becomes $0$ and you are left with Ridge.



                    You can check the loss for Ridge and Lasso in this book and for elastic net in this paper.






                    share|cite|improve this answer









                    $endgroup$













                    • $begingroup$
                      This looks like a good answer but can you edit to include citations for the hyperlinks? Over time, links die.
                      $endgroup$
                      – Sycorax
                      6 hours ago
















                    1












                    $begingroup$

                    Absolutely! The $alpha$ parameter can be adjusted to either fit a Lasso or a Ridge regression (or something in between). Recall that the loss function which Elastic Net minimizes is $$frac{1}{2N}sum^N_{i=1}(y_i-beta_0-x_i^tbeta)^2+lambdasum_{j=1}^p(frac{1}{2}(1-alpha)beta_j^2+alpha|beta_j|).$$
                    Focus on the second big sum (the one multiplied by $lambda$). If you let $alpha=1$, the first term inside this sum becomes $0$, and the whole function becomes exactly the function that Lasso minimizes (or the Lasso loss function). If you let $alpha=0$, the second term becomes $0$ and you are left with Ridge.



                    You can check the loss for Ridge and Lasso in this book and for elastic net in this paper.






                    share|cite|improve this answer









                    $endgroup$













                    • $begingroup$
                      This looks like a good answer but can you edit to include citations for the hyperlinks? Over time, links die.
                      $endgroup$
                      – Sycorax
                      6 hours ago














                    1












                    1








                    1





                    $begingroup$

                    Absolutely! The $alpha$ parameter can be adjusted to either fit a Lasso or a Ridge regression (or something in between). Recall that the loss function which Elastic Net minimizes is $$frac{1}{2N}sum^N_{i=1}(y_i-beta_0-x_i^tbeta)^2+lambdasum_{j=1}^p(frac{1}{2}(1-alpha)beta_j^2+alpha|beta_j|).$$
                    Focus on the second big sum (the one multiplied by $lambda$). If you let $alpha=1$, the first term inside this sum becomes $0$, and the whole function becomes exactly the function that Lasso minimizes (or the Lasso loss function). If you let $alpha=0$, the second term becomes $0$ and you are left with Ridge.



                    You can check the loss for Ridge and Lasso in this book and for elastic net in this paper.






                    share|cite|improve this answer









                    $endgroup$



                    Absolutely! The $alpha$ parameter can be adjusted to either fit a Lasso or a Ridge regression (or something in between). Recall that the loss function which Elastic Net minimizes is $$frac{1}{2N}sum^N_{i=1}(y_i-beta_0-x_i^tbeta)^2+lambdasum_{j=1}^p(frac{1}{2}(1-alpha)beta_j^2+alpha|beta_j|).$$
                    Focus on the second big sum (the one multiplied by $lambda$). If you let $alpha=1$, the first term inside this sum becomes $0$, and the whole function becomes exactly the function that Lasso minimizes (or the Lasso loss function). If you let $alpha=0$, the second term becomes $0$ and you are left with Ridge.



                    You can check the loss for Ridge and Lasso in this book and for elastic net in this paper.







                    share|cite|improve this answer












                    share|cite|improve this answer



                    share|cite|improve this answer










                    answered 6 hours ago









                    BananinBananin

                    1795




                    1795












                    • $begingroup$
                      This looks like a good answer but can you edit to include citations for the hyperlinks? Over time, links die.
                      $endgroup$
                      – Sycorax
                      6 hours ago


















                    • $begingroup$
                      This looks like a good answer but can you edit to include citations for the hyperlinks? Over time, links die.
                      $endgroup$
                      – Sycorax
                      6 hours ago
















                    $begingroup$
                    This looks like a good answer but can you edit to include citations for the hyperlinks? Over time, links die.
                    $endgroup$
                    – Sycorax
                    6 hours ago




                    $begingroup$
                    This looks like a good answer but can you edit to include citations for the hyperlinks? Over time, links die.
                    $endgroup$
                    – Sycorax
                    6 hours ago










                    red4life93 is a new contributor. Be nice, and check out our Code of Conduct.










                    draft saved

                    draft discarded


















                    red4life93 is a new contributor. Be nice, and check out our Code of Conduct.













                    red4life93 is a new contributor. Be nice, and check out our Code of Conduct.












                    red4life93 is a new contributor. Be nice, and check out our Code of Conduct.
















                    Thanks for contributing an answer to Cross Validated!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396748%2fcan-the-alpha-lambda-values-of-a-glmnet-object-output-determine-whether-ridge-o%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    CARDNET

                    Image
                    CARDNET は日本カードネットワークが運営するクレジット決済ネットワーク。日本カードネットワークの略称としてもCARDNETが使用されている。 事件・事故 2017年4月15日午前11時8分からクレジットカードの決済がしづらくなる障害が発生した。障害は同日午後5時18分に復旧した。原因は、複数あるCARDNETセンターの拠点のうち1つで、2重化してあるL3スイッチの片方が故障し、1系統にトラフィックが集中したことによる輻輳が発生したため。対処としては、故障したL3スイッチ交換とシステム再起動が行われた [1] 。 出典 ^ 金子寛人 ( 2017年4月17日 ). “CARDNETのクレジット決済に6時間強障害、原因はL3スイッチ故障”. 日経コンピュータ. 2018年10月18日 閲覧。 この項目は、コンピュータに関連した 書きかけの項目 です。この項目を加筆・訂正などしてくださる協力者を求めています（PJ:コンピュータ/P:コンピュータ）。 この項目は、経済に関連した 書きかけの項目 です。この項目を加筆・訂正などしてくださる協力者を求めています（ポータル 経済学、プロジェクト 経済）。 This page is only for reference, If you need detailed information, please check here
                    Read more

                    Boot-repair Failure: Unable to locate package grub-common:i386

                    Image
                    0 I am on a system dual-booting Windows and Debian (usually using the latter), and a recent Windows update appears to have wreaked havoc on the Debian boot system. I managed to boot Windows again, but not Linux. Following online suggestions, I created a Live USB of Ubuntu, and booted from that. Then I have installed and run the boot-repair utility. The utility instructs me to type several lines including sudo chroot "/mnt/boot-sav/sda6" apt-get purge -y grub*-common grub-common:i386 To which I receive the error E: Unable to locate package grub-common:i386 I am given to understand the issue may have to do with having UEFI vs BIOS (I believe I have UEFI), but do not understand this well. In any case I have subsequently run the boot-repair BootInfo Summary utility which created the following file h...
                    Read more

                    濃尾地震

                    Image
                    濃尾地震 『岐阜市街大地震之図』 歌川国利画 本震 発生日 1891年（明治24年）10月28日 発生時刻 6時38分50秒（JST） 震央 日本 岐阜県本巣郡西根尾村（現・本巣市） 北緯35度35分 東経136度20分（ 地図 ） 規模    マグニチュード（M）8.0 最大震度    震度7： 注1 福井県今立郡鯖江町、愛知県葉栗郡大田島村、東春日井郡勝川村 地震の種類 直下型地震 被害 死傷者数 死者7,273人、負傷者17,175人 注1：当時の震度階級では「激烈」 プロジェクト:地球科学 プロジェクト:災害 テンプレートを表示 濃尾地震 （のうびじしん）は、1891年（明治24年）10月28日に濃尾地方で発生した、日本史上最大の内陸地殻内地震。「 美濃・尾張地震 （みの・おわりじしん）」とも呼ばれている。辛卯の年に発生したことから 辛卯震災 と呼んでいる報告書もある。 目次 1 概要 1.1 震源断層 2 被害 3 各地の震度 4 前兆現象 5 報道 6 学術的な意義 7 地震防災 8 脚注 9 関連項目 10 外部リンク 概要 濃尾地震発生当時の根尾谷断層 濃尾地震を引き起こした根尾谷断層 写真中央を斜めに走る段差が根尾谷断層 濃尾地震は、1891年10月28日6時38分50秒に発生した。震源は、岐阜県本巣郡西根尾村（現・本巣市）、北緯35度35分、東経136度20分付近。河角廣 (1951) は岐阜市付近（北緯35.6度、東経136.6度）に震央を仮定し規模 M K = 7.0 を与え [1] 、マグニチュードは M = 8.4 に換算されているが、明治・大正期の地震については0.5程度大きく見積もられているとされる [2] 。また、震央距離と震度との関係など当時のデータから後にM8.0 [3] とも推定される。アメリカ地質調査所 (USGS) でも8.0としている [4] 。「根尾谷断層帯」が活動した典型的な内陸地殻内地震（いわゆる直下型地震）であり、これは記録が残っている日本の内陸...
                    Read more