Author | Message | Time |
---|---|---|
Adron | Calculate 8000 * 0.3 - 8000 * 3 / 10 Comments? | December 15, 2005, 7:23 PM |
Explicit[nK] | 0? | December 15, 2005, 8:11 PM |
Adron | [quote author=Explicit[nK] link=topic=13532.msg137833#msg137833 date=1134677508] 0? [/quote] What version of VB did you use to come to that conclusion? If you did not, consider why this was posted to the VB forum... ;) | December 15, 2005, 8:45 PM |
Quarantine | I got 0 (VB6) | December 15, 2005, 8:55 PM |
QwertyMonster | I got 0 too (I used Visual basic 6). | December 15, 2005, 8:55 PM |
Explicit[nK] | [quote author=Adron link=topic=13532.msg137841#msg137841 date=1134679503] [quote author=Explicit[nK] link=topic=13532.msg137833#msg137833 date=1134677508] 0? [/quote] What version of VB did you use to come to that conclusion? If you did not, consider why this was posted to the VB forum... ;) [/quote] Pffffft, who needs VB when you have your fingers? I counted out to 8000, and chopped off a finger to attain the three-tenths fraction, then smashed my hands together and found nothing was left. :) | December 15, 2005, 8:57 PM |
Adron | Hmm, weird. I did that in VB6, and did not get 0. Just typed it in the debug window, [code] ?8000 * 0.3 - 8000 * 3 / 10 [/code] | December 15, 2005, 10:09 PM |
Ringo | I got -8.88178419700125E-14 [Edit]: and -1500 in a string | December 15, 2005, 10:45 PM |
Myndfyr | [quote author=Adron link=topic=13532.msg137879#msg137879 date=1134684598] Hmm, weird. I did that in VB6, and did not get 0. Just typed it in the debug window, [code] ?8000 * 0.3 - 8000 * 3 / 10 [/code] [/quote] Perhaps the debug window doesn't follow order of operations? | December 15, 2005, 10:46 PM |
HdxBmx27 | Wow another reason ive droped VB. But one question. WTF!?!?! ~-~(HDX)~-~ | December 16, 2005, 12:25 AM |
Adron | [quote author=MyndFyre link=topic=13532.msg137890#msg137890 date=1134686772] Perhaps the debug window doesn't follow order of operations? [/quote] Well, it originally came from a larger program. So it's not just the debug window. | December 16, 2005, 12:37 AM |
dRAgoN | [quote author=Adron link=topic=13532.msg137826#msg137826 date=1134674620] Calculate 8000 * 0.3 - 8000 * 3 / 10 Comments? [/quote] [code]'VB6 Private Sub Form_Load() Dim a As Double Dim b As Double Dim x As Double Dim y As Double a = 8000 * 0.3 '2400 b = 8000 * 3 / 10 '2400 x = a - b y = 8000 * 0.3 - 8000 * 3 / 10 Debug.Print x 'Real answer '= 0 Debug.Print y 'Real Problem '= -8.88178419700125E-14 End Sub[/code] Is this problem only with subtraction? (Addition works fine.) I guess the solution to this is to create your own calculation processor. | December 16, 2005, 4:35 AM |
Ringo | Currency can do it all in one: Const Zero As Currency = 8000 * 0.3 - 8000 * 3 / 10 | December 16, 2005, 5:10 AM |
Adron | [quote author=Ringo link=topic=13532.msg137976#msg137976 date=1134709859] Currency can do it all in one: Const Zero As Currency = 8000 * 0.3 - 8000 * 3 / 10 [/quote] Currency can't handle the size of the answer so it rounds to zero. I'll give you another test for that though: Const Answer As Currency = Fix(8000 * 0.3) | December 16, 2005, 6:00 AM |
Ringo | hmm, what about CDec(CDbl(8000 * 0.3) - CDbl(8000 * 3 / 10))? | December 16, 2005, 9:28 AM |
Myndfyr | You don't suppose VB uses its own floating-point library from the days that most computers didn't have coprocessors, do you? | December 16, 2005, 3:59 PM |
l2k-Shadow | [code] Sub form_load() Dim a#, b#, c#, d# a = (8000 * 0.3) b = 8000 * 3 c = b / 10 d = a - c Text1.Text = d 'Answer = 0 End Sub [/code] | December 18, 2005, 1:18 AM |
FrOzeN | [quote author=l2k-Shadow link=topic=13532.msg138245#msg138245 date=1134868680] I just used [code] Sub form_load() Dim a As Double a = 8000 * 0.3 - 8000 * 3 / 10 Text1.Text = a End Sub [/code] and got -8.88178419700125E-14 .. what's the problem? [/quote] The problem is that it returns the wrong answer, it should be 0. | December 18, 2005, 1:22 AM |
l2k-Shadow | [quote author=FrOzeN link=topic=13532.msg138249#msg138249 date=1134868960] [quote author=l2k-Shadow link=topic=13532.msg138245#msg138245 date=1134868680] I just used [code] Sub form_load() Dim a As Double a = 8000 * 0.3 - 8000 * 3 / 10 Text1.Text = a End Sub [/code] and got -8.88178419700125E-14 .. what's the problem? [/quote] The problem is that it returns the wrong answer, it should be 0. [/quote] Yeah I just realized that, edited my post. | December 18, 2005, 1:24 AM |
111787 | Question: What order does it do it in? I cant produce that answer doing it in any order. | December 18, 2005, 3:10 AM |
Adron | [quote author=111787 link=topic=13532.msg138267#msg138267 date=1134875405] Question: What order does it do it in? I cant produce that answer doing it in any order. [/quote] Regular math evaluation order... And copying the expression exactly as I typed it should give the nonzero result. What I was originally doing was take 30% of an input value, and round it down to the nearest integer, i.e. Fix(x * 0.30), and this turned out to be 2399 for the input 8000. Which I thought was kinda odd... | December 18, 2005, 6:52 AM |
laurion | which would be Parenthesees Exponents Multiplication Division Addition Subtraction | December 18, 2005, 2:17 PM |
rabbit | [quote author=Adron link=topic=13532.msg138288#msg138288 date=1134888756] [quote author=111787 link=topic=13532.msg138267#msg138267 date=1134875405] Question: What order does it do it in? I cant produce that answer doing it in any order. [/quote] Regular math evaluation order... And copying the expression exactly as I typed it should give the nonzero result. What I was originally doing was take 30% of an input value, and round it down to the nearest integer, i.e. Fix(x * 0.30), and this turned out to be 2399 for the input 8000. Which I thought was kinda odd... [/quote]1/3 is 0.333333 | December 18, 2005, 8:39 PM |
Explicit[nK] | [quote author=rabbit link=topic=13532.msg138327#msg138327 date=1134938359] [quote author=Adron link=topic=13532.msg138288#msg138288 date=1134888756] [quote author=111787 link=topic=13532.msg138267#msg138267 date=1134875405] Question: What order does it do it in? I cant produce that answer doing it in any order. [/quote] Regular math evaluation order... And copying the expression exactly as I typed it should give the nonzero result. What I was originally doing was take 30% of an input value, and round it down to the nearest integer, i.e. Fix(x * 0.30), and this turned out to be 2399 for the input 8000. Which I thought was kinda odd... [/quote]1/3 is 0.333333 [/quote] Where are you even getting 1/3 from? | December 18, 2005, 9:54 PM |
rabbit | I have no clue...I guess I was spacing out.. O well, it was a random fun fact I guess :P | December 19, 2005, 12:30 AM |
Explicit[nK] | [quote author=rabbit link=topic=13532.msg138378#msg138378 date=1134952253] I have no clue...I guess I was spacing out.. O well, it was a random fun fact I guess :P [/quote] Silly rabbit, fun facts are for kids! :) | December 19, 2005, 12:32 AM |
rabbit | Wow. That wasn't funny the first time. Nor the second time. Why the hell would it be funny the seven hundred twenty first time? | December 19, 2005, 4:16 PM |
iNsaNe | How is this hard. The equation is basically the same on both sides. The rule is PEMDAS stated above, Parenthesis, Exponents, Multiplication OR Division (Left to Right), Addition OR Subtraction(Left to Right). The original equation: 8000 × 0.3 - 8000 × 3 / 10 - Since PEMDAS goes in order from Multiplication OR division from Left to Right, the "× 3 / 10" means × 0.3 The equation basically says: = (8000 × 0.3) - (8000 × 0.3) = 0 Why is it necessary to go in such depth? | March 5, 2006, 10:37 PM |
l2k-Shadow | [quote author=iNsaNe link=topic=13532.msg147657#msg147657 date=1141598277] How is this hard. The equation is basically the same on both sides. The rule is PEMDAS stated above, Parenthesis, Exponents, Multiplication OR Division (Left to Right), Addition OR Subtraction(Left to Right). The original equation: 8000 × 0.3 - 8000 × 3 / 10 - Since PEMDAS goes in order from Multiplication OR division from Left to Right, the "× 3 / 10" means × 0.3 The equation basically says: = (8000 × 0.3) - (8000 × 0.3) = 0 Why is it necessary to go in such depth? [/quote] don't bring up 1 month+ old topics please | March 5, 2006, 10:44 PM |