RSS

Category Archives: Machine Learning

Binary cross entropy and cross entropy loss usage in PyTorch

reference
https://discuss.pytorch.org/t/lstm-crossentropyloss-change-to-bceloss/5320/4

t1 = torch.randn(10, 2)
print ('t1 : ', t1)
score = Variable(t1)
t2 = torch.rand(10) > 0.5
print ('t2 : ', t2)
t3 = t2.long()
print ('t3 : ', t3)
target = Variable(t3)
lfn1 = torch.nn.CrossEntropyLoss()
lfn2 = torch.nn.BCELoss()
t4 = lfn1(score, target)
print('t4 : ', t4)
t5 = torch.nn.functional.softmax(score)
print ('t5 : ', t5)
t6 = t5[:, 1]
print ('t6 : ', t6)
t7 = target.float()
print ('t7 : ', t7)
t8 = lfn2(t6, t7)
print('t8 : ', t8)t.float()))

결과

t1 :
-0.6985 0.4857
-0.4547 0.8040
-1.4664 1.2185
0.5449 0.0662
0.7061 -0.6480
-0.8922 0.4205
0.1444 -0.1308
1.4784 0.7342
0.6642 -0.3723
-0.9741 -0.3100
[torch.FloatTensor of size 10×2]

t2 :
1
1
0
0
0
0
1
1
0
1
[torch.ByteTensor of size 10]

t3 :
1
1
0
0
0
0
1
1
0
1
[torch.LongTensor of size 10]

t4 : Variable containing:
0.8223
[torch.FloatTensor of size 1]

t5 : Variable containing:
0.2343 0.7657
0.2212 0.7788
0.0639 0.9361
0.6174 0.3826
0.7948 0.2052
0.2120 0.7880
0.5684 0.4316
0.6779 0.3221
0.7382 0.2618
0.3398 0.6602
[torch.FloatTensor of size 10×2]

t6 : Variable containing:
0.7657
0.7788
0.9361
0.3826
0.2052
0.7880
0.4316
0.3221
0.2618
0.6602
[torch.FloatTensor of size 10]

t7 : Variable containing:
1
1
0
0
0
0
1
1
0
1
[torch.FloatTensor of size 10]

t8 : Variable containing:
0.8223
[torch.FloatTensor of size 1]

The results of [CrossEntropyLoss] and [softmax + BCELoss] are the same, which means CrossEntropyLoss includes softmax in it.

Advertisements
 
Leave a comment

Posted by on March 13, 2018 in PyTorch

 
Link

Comparing a Random Forest to a CART model (Part 2)

 

 
Leave a comment

Posted by on June 27, 2014 in Machine Learning

 
Link

https://github.com/rasmusbergpalm/DeepLearnToolbox

 
Leave a comment

Posted by on June 18, 2014 in Deep Learning

 
Link

http://kourouklides.wikia.com/wiki/Deep_Learning#Books_and_Book_Chapters

 
Leave a comment

Posted by on June 18, 2014 in Deep Learning

 
Link

http://www.pyimagesearch.com/2014/06/09/get-deep-learning-bandwagon-get-perspective

 
Leave a comment

Posted by on June 18, 2014 in Deep Learning

 
Link

https://code.google.com/p/visual-rbm/

 
Leave a comment

Posted by on June 17, 2014 in Deep Learning, Programming

 

[term] latent

2014.06.05

현재까지는 latent를 hidden이란 말과 비슷한 의미로 이해하고 있음.  특히 HMM에서의 hidden이 가진 의미.  또한 latent variable은 model parameter와 비슷한 느낌으로 받아들이고 있음.

 
Leave a comment

Posted by on June 5, 2014 in Machine Learning, Terms