Sampling from Gaussian mixture models, when are the sampled data independent?












2












$begingroup$


Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions



$p(x) = sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)$



where $w_i$ are the weights.



Now I sample some points ${x_n}$ from $p(x)$



What condition is needed to ensure that the points are independent from each other?



Is it sufficient to assume that each component $mathcal{N}(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?






normal-distribution mixed-model random-variable independence gaussian-mixture







share|cite|improve this question









$endgroup$

















    2












    $begingroup$


    Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions



    $p(x) = sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)$



    where $w_i$ are the weights.



    Now I sample some points ${x_n}$ from $p(x)$



    What condition is needed to ensure that the points are independent from each other?



    Is it sufficient to assume that each component $mathcal{N}(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?






    normal-distribution mixed-model random-variable independence gaussian-mixture







    share|cite|improve this question









    $endgroup$















      2












      2








      2





      $begingroup$


      Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions



      $p(x) = sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)$



      where $w_i$ are the weights.



      Now I sample some points ${x_n}$ from $p(x)$



      What condition is needed to ensure that the points are independent from each other?



      Is it sufficient to assume that each component $mathcal{N}(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?






      normal-distribution mixed-model random-variable independence gaussian-mixture







      share|cite|improve this question









      $endgroup$




      Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions



      $p(x) = sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)$



      where $w_i$ are the weights.



      Now I sample some points ${x_n}$ from $p(x)$



      What condition is needed to ensure that the points are independent from each other?



      Is it sufficient to assume that each component $mathcal{N}(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?






      normal-distribution mixed-model random-variable independence gaussian-mixture




      normal-distribution mixed-model random-variable independence gaussian-mixture






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked 8 hours ago









      Shamisen ExpertShamisen Expert

      126112




      126112






















          1 Answer
          1






          active

          oldest

          votes


















          5












          $begingroup$

          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            8 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            1 hour ago













          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "65"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396582%2fsampling-from-gaussian-mixture-models-when-are-the-sampled-data-independent%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          5












          $begingroup$

          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            8 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            1 hour ago


















          5












          $begingroup$

          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            8 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            1 hour ago
















          5












          5








          5





          $begingroup$

          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.






          share|cite|improve this answer











          $endgroup$



          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.







          share|cite|improve this answer














          share|cite|improve this answer



          share|cite|improve this answer








          edited 7 hours ago

























          answered 8 hours ago









          Xi'anXi'an

          57.9k897360




          57.9k897360












          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            8 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            1 hour ago




















          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            8 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            1 hour ago


















          $begingroup$
          Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
          $endgroup$
          – Shamisen Expert
          8 hours ago




          $begingroup$
          Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
          $endgroup$
          – Shamisen Expert
          8 hours ago












          $begingroup$
          Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
          $endgroup$
          – mb7744
          1 hour ago






          $begingroup$
          Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
          $endgroup$
          – mb7744
          1 hour ago




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Cross Validated!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396582%2fsampling-from-gaussian-mixture-models-when-are-the-sampled-data-independent%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          CARDNET

          Image
          CARDNET は日本カードネットワークが運営するクレジット決済ネットワーク。日本カードネットワークの略称としてもCARDNETが使用されている。 事件・事故 2017年4月15日午前11時8分からクレジットカードの決済がしづらくなる障害が発生した。障害は同日午後5時18分に復旧した。原因は、複数あるCARDNETセンターの拠点のうち1つで、2重化してあるL3スイッチの片方が故障し、1系統にトラフィックが集中したことによる輻輳が発生したため。対処としては、故障したL3スイッチ交換とシステム再起動が行われた [1] 。 出典 ^ 金子寛人 ( 2017年4月17日 ). “CARDNETのクレジット決済に6時間強障害、原因はL3スイッチ故障”. 日経コンピュータ. 2018年10月18日 閲覧。 この項目は、コンピュータに関連した 書きかけの項目 です。この項目を加筆・訂正などしてくださる協力者を求めています（PJ:コンピュータ/P:コンピュータ）。 この項目は、経済に関連した 書きかけの項目 です。この項目を加筆・訂正などしてくださる協力者を求めています（ポータル 経済学、プロジェクト 経済）。 This page is only for reference, If you need detailed information, please check here
          Read more

          Boot-repair Failure: Unable to locate package grub-common:i386

          Image
          0 I am on a system dual-booting Windows and Debian (usually using the latter), and a recent Windows update appears to have wreaked havoc on the Debian boot system. I managed to boot Windows again, but not Linux. Following online suggestions, I created a Live USB of Ubuntu, and booted from that. Then I have installed and run the boot-repair utility. The utility instructs me to type several lines including sudo chroot "/mnt/boot-sav/sda6" apt-get purge -y grub*-common grub-common:i386 To which I receive the error E: Unable to locate package grub-common:i386 I am given to understand the issue may have to do with having UEFI vs BIOS (I believe I have UEFI), but do not understand this well. In any case I have subsequently run the boot-repair BootInfo Summary utility which created the following file h
          Read more

          濃尾地震

          Image
          濃尾地震 『岐阜市街大地震之図』 歌川国利画 本震 発生日 1891年（明治24年）10月28日 発生時刻 6時38分50秒（JST） 震央 日本 岐阜県本巣郡西根尾村（現・本巣市） 北緯35度35分 東経136度20分（ 地図 ） 規模    マグニチュード（M）8.0 最大震度    震度7： 注1 福井県今立郡鯖江町、愛知県葉栗郡大田島村、東春日井郡勝川村 地震の種類 直下型地震 被害 死傷者数 死者7,273人、負傷者17,175人 注1：当時の震度階級では「激烈」 プロジェクト:地球科学 プロジェクト:災害 テンプレートを表示 濃尾地震 （のうびじしん）は、1891年（明治24年）10月28日に濃尾地方で発生した、日本史上最大の内陸地殻内地震。「 美濃・尾張地震 （みの・おわりじしん）」とも呼ばれている。辛卯の年に発生したことから 辛卯震災 と呼んでいる報告書もある。 目次 1 概要 1.1 震源断層 2 被害 3 各地の震度 4 前兆現象 5 報道 6 学術的な意義 7 地震防災 8 脚注 9 関連項目 10 外部リンク 概要 濃尾地震発生当時の根尾谷断層 濃尾地震を引き起こした根尾谷断層 写真中央を斜めに走る段差が根尾谷断層 濃尾地震は、1891年10月28日6時38分50秒に発生した。震源は、岐阜県本巣郡西根尾村（現・本巣市）、北緯35度35分、東経136度20分付近。河角廣 (1951) は岐阜市付近（北緯35.6度、東経136.6度）に震央を仮定し規模 M K = 7.0 を与え [1] 、マグニチュードは M = 8.4 に換算されているが、明治・大正期の地震については0.5程度大きく見積もられているとされる [2] 。また、震央距離と震度との関係など当時のデータから後にM8.0 [3] とも推定される。アメリカ地質調査所 (USGS) でも8.0としている [4] 。「根尾谷断層帯」が活動した典型的な内陸地殻内地震（いわゆる直下型地震）であり、これは記録が残っている日本の内陸
          Read more